ISSN  0316-6295 


A  Panache  of  DIMS  Ideas  II 


edited  by 


F  .E  . . Lochovsky 


Technical  Report  CSRG-121 
May,  1979 

_ .  _ J 


COMPUTER  SYSTEMS  RESEARCH  GROUP 

UNIVERSITY  OF  TORONTO 


IftR.BO'gJ 


A  Panache  of  DIMS  Ideas  II 


edited  by 
F .E . . Lochovsky 


Technical  Report  CSRG-101 
May,  1279 


Computer  Systems  Research  Group 
University  of  Toronto 
Toronto,  Ontario 
Canada,  M5S  1A4 


The  Computer  Systems  Research  Group  (CSRG)  is  an 
interdisciplinary  group  formed  to  conduct  research  and 
development  relevart  to  computer  systems  and  their  application. 
It  is  .jointly  administered  ty  the  Department  of  Electrical 
Engineering  and  the  Department  of  Computer  Science  of  the 
University  of  Toronto,  and  is  supported  in  part  by  the  Natural 
Sciences  and  Engineering  Council  of  Canada. 


C  1272,  Computer  Systems  Research  Group,  University  of  Toronto. 


Digitized  by  the  Internet  Archive 
in  2018  with  funding  from 
University  of  Toronto 


https://archive.org/details/technicalreportc101univ 


Ats tract 


This  report  is  a  collection  of  papers  outlining  some  of  the 
research  work  carried.  out  hy  the  Data  Ease  Group  within  the 
Computer  Systems  Research  group  at  the  University  of  Toronto. 
The  papers  detail  the  research  carried  out  since  the  publication 
of  the  first  panache  (CSRG-78)  in  February  1977.  As  with  the 
first  panache,  there  is  no  common  theme  to  the  report.  It  is 
just  a  collection,  a  mix  (panache)  of  different  ideas  by  a 
tightly  knit  group  of  people. 
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1  •  Introduction 

In  the  "good  old  times"  computers  were  really  expensive. 
Their  cost  overshadowed  any  other  expense.  Their  users  were  few 
and  very  knowledgeable.  Their  priests  were  very  talented.  Any 
idi osyn chrancy  in  their  operation  was  attributed  to  efficiency 
goals  in  their  use.  Their  applications  were  well  understood,  We 
could  afford  to  waste  some  systems  programmers'  creative  talents 
to  babysit  computers  and  bend  people's  needs  and  tastes  to  accom¬ 
modate  machines.  Unfortunately  computers  are  getting  cheaper. 
While  this  solves  many  old  problems  it  creates,  or  rather  makes 
visible,  some  rather  nasty  new  ones.  We  were  getting  used  to 
solving  the  old  problems.  Unfortunately,  their  solutions  do  not 
provide  us  with  any  insight  about  handling  the  new  ones.  The  old 
problems  were  mainly  machine  problems.  They  were  better  defined. 
The  new  problems  are  people  plus  machine  problems.  They  are  dif¬ 
ficult  to  pin  down,  let  alone  solve.  We  will  soon  have  the  capa¬ 
bility  of  putting  a  phenomenal  amount  of  computing  power  at 
everybody's  disposal  for  a  small  cost.  The  problem  is  to  explore 
ways  that  this  power  can  be  used  effectively. 

When  computers  were  very  expensive,  their  use  was  carefully 
controlled  and  their  resource  allocation  heavily  optimized.  The 
computer  service  was  centered  around  a  general  purpose  computer. 
Extensive  floor  space  and  manpower  was  committed  to  the  transla¬ 
tion  of  raw  computer  power  into  very  different  services.  Many 
diversified  applications  were  forcibly  brought  together  because 
they  had  to  share  a  common  machine.  As  a  result,  computer 
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centers  "became  an  all-encompassing  tool  "but  also  a  major  "burden. 

On  the  other  hand,  if  computers  are  cheap,  they  can  proli¬ 
ferate.  They  can  migrate  exactly  where  they  are  needed.  Dupli¬ 
cation  of  hardware  is  not  a  major  obstacle  any  more.  It  is  more 
important  to  optimize  the  use  of  personnel  and  floor  space. 
There  is  no  need  to  try  to  understand  everybody's  problems  in 
order  to  solve  them  with  the  big  machine.  If  we  provide  the  end 
users  with  the  right  tools,  they  can  determine  the  best  way  to 
deal  with  their  problems. 

The  low  cost  of  computers  today  allows  us  to  break  up  gen¬ 
eral  purpose  computer  systems  into  smaller  functional  units. 
However,  it  is  often  very  difficult  to  decide  what  these  func¬ 
tional  pieces  should  be.  We  should  not  simply  look  inside 
present  computer  systems  and  try  to  isolate  their  functional 
modules,  e.g.,  compilers,  data  base  systems,  etc.  That  may  point 
us  in  the  wrong  direction.  Present  computer  systems  were 
designed  to  serve  a  specific  environment.  Since  the  environment 
has  changed  they  themselves  and  their  function  is  subject  to 
change.  We  should  look  instead  to  the  end  uses  of  computers  and 
investigate  their  different  characteristics.  We  are  not  in¬ 
terested  in  describing  the  exact  way  that  computer  services 
should  be  provided.  There  are  a  wide  range  of  solutions  accord¬ 
ing  to  different  needs  and  operational  environments.  We  are  in¬ 
terested  only  in  the  directions  that  the  solutions  may  take  and 
the  kind  of  systems  we  may  end  up  with.  This  in  turn  has  some 
relevance  to  the  kind  of  software  we  will  need. 
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2.  Computer  usage 

As  a  first  step,  we  can  classify  computer  services  according 
to  two  groups:  services  to  people  of  an  organization  and  services 
to  the  organization's  needs.  Most  people  in  an  organization  have 
simple  needs  usually  provided  by  office  systems.  They  need  to 
process  documents,  to  have  access  to  design  and  analysis  tools, 
to  store  and  retrieve  data,  etc.  We  will  refer  to  these  needs  as 
personal  computing  services.  Note  that  personal  computing  ser¬ 
vices  have  little  to  do  with  computation.  Very  few  people  need 
computational  power  which  can  not  be  provided  by  a  cheap  calcula¬ 
tor.  Personal  computing  services  deal  mainly  with  information 
processing.  That  is,  they  provide  a  flexible  and  easy  to  use  en¬ 
vironment  to  obtain,  store,  digest  and  create  information.  Per¬ 
sonal  computing  services  manipulate  data  which  is  to  a  large  ex¬ 
tent  local.  They  produce  results  which  have  for  the  most  part 
local  flavor  and  value.  For  example,  text  editors,  interactive 
packages,  etc.  are  used  for  personal  computing  services. 

The  needs  of  the  organization  are  provided  by  organization 
computing  services.  These  are  services  which  work  on  large  sets 
of  global  data.  They  produce  results  which  benefit  the  organiza¬ 
tion  as  a  whole.  These  results  may  be  geared  for  a  few  people, 
e.g.,  management.  However,  these  people  have  access  to  the 
results  not  as  people  but  as  actors  playing  a  role  in  the  organi¬ 
zation,  Their  needs  and  the  benefit  of  their  actions  are  related 
to  the  organization  as  a  whole.  For  example,  payroll  systems, 
inventory  control  systems,  etc.  provide  organization  computing 
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services . 

Our  distinction  "between  personal  and  organization  Computing 
services  is  necessarily  blurred.  Afterall,  if  we  serve  the 
organization's  people  we  serve  the  organization  and  vice  versa 
(hopefully).  The  distinction  can  be  clarified  by  a  hypothetical 
situation.  Suppose  we  place,  temporarily,  the  entire  computer 
center  of  an  organization  at  an  average  person's  disposal.  Any¬ 
thing  he  would  require  in  order  to  better  perform  his/her  job  we 
call  personal  computing  services.  Anything  else  that  the  comput¬ 
er  center  does  for  the  benefit  of  the  organization  is  organiza¬ 
tion  computing  services. 

There  is  a  third  area  where  personal  and  organization  com¬ 
puting  services  blend  together.  This  area  will  be  loosely  called 
interaction  services.  People  use  interaction  services  to  commun¬ 
icate  with  other  people.  These  services  satisfy  personal  needs. 
They  can,  therefore,  be  classified  as  personal  computing  ser¬ 
vices.  They  are  provided,  however,  in  a  global  fashion.  They, 
therefore,  can  be  thought  of  also  as  organization  services.  As  a 
matter  of  fact,  interaction  services  can  be  provided  in  two  dis¬ 
tinct  ways.  In  one  way,  we  have  a  global  data  base  where  users 
can  deposit  or  retrieve  information.  Using  the  common  data  base 
they  can  interact  heavily  with  each  other.  This  is  more  of  an 
organization  computing  service.  The  common  information  can  also 
be  used  for  other  global  processing.  In  the  other  way,  we  can 
have  a  message  system  which  users  use  to  send  messages  to  each 
other.  This  is  more  of  a  personal  service.  The  user  views  this 
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service  as  mainly  satisfying  his  needs  and  not  any  global  organi¬ 
zation  necessity.  However,  message  systems  are  provided  in  a  un¬ 
iform  manner  as  a  tool  for  the  organization  as  a  whole. 

Personal  computing  services,  organization  computing  services 
and  interaction  services  can  he  provided  all  together  in  terms  of 
a  general-purpose,  time-sharing  system.  The  user  can  access  the 
system  through  a  terminal  for  personal  computing  services.  Glo¬ 
bal  organization  needs  are  satisfied  by  the  same  centralized  sys¬ 
tem.  Finally,  interaction  services  are  provided  through  process 
communication  and  file  sharing.  This  combination  of  different 
services  is  dictated  by  the  cost  of  hardware  and  the  need  for  its 
efficient  utilization. 

In  general  purpose  systems  there  is  a  distinction  between 
batch  and  on-line  systems.  This  distinction  deals  mainly  with 
the  way  the  services  are  provided  and  not  their  function.  Batch 
jobs  relate  mainly  to  organization  computing  services.  A  person, 
however,  may  submit  a  problem  analysis  as  a  batch  job.  Hence, 
some  batch  jobs  can  correspond  to  personal  computing  services. 
On-line  activity  has  three  components.  It  has  a  personal  ser¬ 
vices  component  since  that  is  the  only  way  that  some  personal 
services  can  be  offered,  e.g.,  a  person  doing  text  editing.  On¬ 
line  systems  provide  the  means  for  heavy  interaction  between 
users.  Finally,  on-line  systems  have  some  organization  services 
which  can  also  be  thought  of  as  interaction  services.  For  in¬ 
stance,  a  reservation  transaction  benefits  the  organization  but 
basically  provides  an  interaction  facility.  One  of  the  reasons 
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that  general  purpose  systems  are  so  complex  is  that  they  mix  dif¬ 
ferent  kinds  of  services.  A  second  reason  is  that  they  try  to 
provide  all  these  diversified  services,  while  optimizing  their 
resource  allocation.  Actually,  the  optimization  of  their 
resources  is  a  fourth  and  major  service  which  they  provide  for 
themselves  and  which  competes  with  the  other,  user  oriented,  ser¬ 
vices. 

If  the  cost  of  hardware  is  not  an  overriding  factor,  ser¬ 
vices  can  he  unbundled.  It  seems  reasonable  to  separate  personal 
computing  services  and  provide  them  by,  what  else,  personal  com¬ 
puters.  Organization  computing  services  can  be  retained  by  large 
centralized  systems.  Interaction  services  can  be  provided  by  a 
combination  of  both  centralized  and  decentralized  systems.  As  a 
matter  of  fact,  interaction  services  now  become  very  important. 
They  are  no  longer  implicit  and  provided  de  facto  due  to  the  ex¬ 
istence  of  a  common  host.  They  are  explicit  and  need  special 
hardware,  in  terms  of  networks,  and  software,  in  terms  of  message 
systems.  On  the  other  hand,  interaction  is  purely  positive.  It 
does  not  imply  interference  or  unwanted  interaction  forced  by  the 
use  of  common  computing  resources.  It  is  interaction,  when  and 
if,  the  users  want  it. 

3.  Personal  computing  services 

In  the  previous  section  we  separated,  conceptually,  personal 
computing  services  and  argued  that  they  can  be  provided  in  a  de¬ 
centralized  fashion.  First,  let  us  observe  that  a  decentralized 
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system  can  he  visualized  as  the  result  of  breaking  up  and  distri¬ 
buting  some  of  the  resources  of  a  centralized  system  among  the 
users'  stations.  This  operation  is  carried  out  so  as  to  com¬ 
pletely  separate  the  resources  needed  for  personal  services.  To 
propose  this  decentralization  as  a  realistic  solution  we  should 
think  about  individual  components  of  the  centralized  system  and 
argue  the  merits  of  their  distribution.  We  will  discuss,  there¬ 
fore,  decentralization  of  four  types  of  resources  -  processors, 
main  memory,  secondary  storage  and  peripherals.  In  addition,  we 
will  comment  on  the  effects  of  the  decentralization  on  communica¬ 
tions,  coordination  and  control,  personnel  and  services,  and  fi¬ 
nally  the  real  target  of  this  paper  software. 

In  a  centralized  system  a  large  processor  is  time  sliced 
among  the  users.  The  processor  allocation  is  very  dynamic  and 
flexible.  In  the  decentralized  system  the  processing  power  is 
statically  allocated  and  very  Inflexible.  With  microprocessors, 
however,  it  makes  sense  to  completely  separate  the  processors. 
We  lose  in  flexibility  and  availability  of  a  larger  processing 
unit.  However,  we  gain  in  simplicity  of  operation.  In  addition, 
processing  is  getting  so  cheap  that  underutilizing  it  is  not  that 
critical.  The  only  real  waste  is  that  the  personal  processors  at 
the  user's  site  are  completely  idle  when  the  user  does  not  use 
them.  This  can  be  remedied  by  providing  real  microprocessors  (as 
opposed  to  virtual  machines)  to  a  user  via  a  switching  system. 
Alternatively,  in  a  room  of  not  so  personal  microprocessors, 
users  can  walk  in  and  use  an  available  one.  In  summary,  process- 
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ing  power  can  he  provided  for  personal  services  hy  using  personal 
computers,  as  opposed  to  virtual  ones  through  a  large  time-shared 
sys  tern. 

Memory  is  more  expensive  than  processing.  We  can,  of 
course,  centralize  the  memory  and  dynamically  allocate  it  to  a 
set  of  processors.  However,  memory  itself  is  getting  cheap 
enough  that  we  do  not  need  to  worry  too  much  about  its  utiliza¬ 
tion.  It  is  far  simpler  and  more  natural  to  associate  the  memory 
with  each  processor.  We  end  up  with  a  network  of  real  processors 
with  real  memory  connected  together.  Notice  that  hy  memory  we 
mean  any  kind  of  memory  which  is  not  rotating  storage.  To  the 
extent  that  monolithic  memory,  bubbles  or  CCD's  become  realistic 
alternatives  for  providing  inexpensive  Mbytes  of  memory,  we  will 
look  on  them  as  memory  to  be  associated  with  a  personal  proces¬ 
sor  . 


By  secondary  storage  we  imply  disk  type  storage.  Disks  are 
not  cheap  enough  to  distribute  widely.  We  can  argue  that  they 
will  never  be  too  cheap,  because  they  are  sensitive  elec¬ 
tromechanical  devices.  In  addition,  the  cost  per  byte  of  large 
disks  is  cheaper  than  small  disks.  So  we  have  at  least  one  kind 
of  device  that  should  not  be  widely  distributed.  Disks  do  not 
work  by  themselves.  We  need  some  processing  and  memory  to  drive 
them.  A  small  computer  with  a  large  disk  can  provide  disk  space 
for  many  users.  In  this  way,  disk  space  is  clustered  and  can  be 
used  by  many  personal  computers. 
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Other  peripherals  are  also  clustered.  We  do  not  envision  a 
printer  in  everybody's  office.  They  are  not  only  expensive,  hut 
bothersome.  Printers  should  be  where  people  find  easy  to  access 
them.  For  example,  in  a  room  within  easy  walking  distance.  In 
addition,  there  may  be  other  not  commonly  used  devices  which  may 
be  clustered  and  shared  among  users. 

The  communication  among  all  these  machines  can  be  via  local 
networks,  global  networks,  multiplexed  telephone  lines,  fiber  op¬ 
tics,  etc.  Communications  are  evolving  so  rapidly  that  they  pro¬ 
vide  us  with  many  exciting  alternatives.  There  are  no  binding 
restrictions  in  terms  of  communications. 

Maintenance  of  a  decentralized  system  should  pose  no  great 
difficulty.  Suppose  we  are  dealing  wi-th  a  homogeneous  system  of 
personal  computers  using  common  software.  It  is  no  harder  to 
service  both  the  hardware  and  the  software  in  this  system  then  in 
a  centralized  operation.  Personal  computers  should  not  require 
any  more  hardware  maintenance  than  terminals.  The  common 
software  can  be  maintained  centrally.  There  is  no  need  for 
operators,  except  perhaps  for  the  printer  clusters  and  the  disk 
storage  machines. 

Our  arguments  point  to  a  system  architecture  for  personal 
computing  services  where  a  number  of  small  personal  processors 
with  their  own  memory  are  tied  together  through  a  network.  In 
addition,  there  are  common  processors  that  manage  disk  storage 
and  other  peripherals.  An  environment  of  this  kind  exists  at 
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Xerox  PARC. 

We  can  not  claim  at  this  point  that  such  a  system  will  pro¬ 
vide  cheaper  personal  computing  services  than  a  time-sharing  sys¬ 
tem.  The  cost  differential,  however,  is  not  overriding.  In  such 
a  case,  the  feeling  of  ownership  and  freedom  from  arbitrary  com¬ 
puter  center  actions  provided  by  a  decentralized  system  will  jus¬ 
tify  the  cost  difference.  As  an  analogy,  cars  are  not  the 
cheapest  way  for  people  to  move  around.  However,  people  will  use 
them  as  long  as  they  are  more  convenient  and  not  unreasonably 
more  expensive  than  public  transportation.  In  the  long  run  per¬ 
sonal  computers  will  win  even  the  economic  argument  for  another 
reason.  They  will  become  cheaper  simply  because  they  will  be 
produced  in  much  larger  volumes  than  large  machines. 

We  will  now  concentrate  on  the  software  directions  in  such 
an  environment  of  personal  computing  services.  The  hardware  ar¬ 
chitecture  of  individual  machines  can  be  assumed  to  be  basically 
similar.  This  implies  that  there  is  no  need  to  change  any  of  the 
system  software  running  on  the  personal  computers.  This  is  true 
for  individual  software  components  like  compilers,  editors,  etc. 
However,  the  environment,  use  and  priorities  of  the  system  are 
quite  different.  As  a  result,  the  software  directions  are  very 
different. 

First  of  all,  there  is  no  need  anywhere  for  multiprogram¬ 
ming.  The  personal  computers  are  certainly  uni  programmed .  The 
disk  and  peripheral  servers  are  also  uniprogrammed .  An  environ- 
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ment  of  un iprogrammed  machines  implies  great  changes  in  the  sys¬ 
tem  software,  especially  in  operating  systems.  There  is  no  need 
for  complicated  scheduling.  There  is  no  need  for  elaborate  pro¬ 
cess  switching.  There  is  no  need  for  complex  memory  allocation. 
All  the  painstaking  work  and  experience  on  complicated  resource 
allocation  algorithms  in  a  mult iorogrammed  system  becomes  largely 
irrelevant.  There  are  many  other  side  effects  of  uniprogramming 
in  terms  of  security,  concurrency,  back-up  strategies,  etc.  The 
major  portion  of  current  centralized  operating  systems  software 
is  no  longer  required.  There  is  also  a  multiplier  effect  on  sim¬ 
plification.  Current  operating  systems  resemble  bridges?  most  of 
their  weight  is  there  to  support  themselves. 

Another  direction  for  software  in  this  environment  is  the 
necessity  for  a  flexible  and  easy  to  use  user  interface.  The 
user  is  now  alone  with  his  own  personal  computer.  He  does  not 
have  direct  access  to  system  or  application  programmers.  In  ad¬ 
dition,  he  is  probably  not  very  sophisticated.  This  points  to  a 
new  environment  of  computer  programming.  Not  only  the  language 
but  the  complete  set  of  tools  at  the  user's  disposal,  editors, 
graphics,  design  tools,  etc.  should  be  designed  for  ease  of  use. 
The  emphasis  should  be  on  end-user  tools  and  not  all-encompassing 
tools  for  the  sophisticated  programmer.  In  addition,  we  can  af¬ 
ford  to  consume  great  amounts  of  processing  power  to  make  this 
interface  friendly.  Artificial  Intelligence  techniques  which 
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were  forbidding  in  the  past  rray  now  "be  appropriate. 

4.  Organization  computing  services 

Organization  computing  services  do  not  seem  to  "be  affected 
as  much  by  hardware  changes.  The  investment  in  terms  of  software 
is  so  massive  that  changes  due  to  hardware  can  only  be  gradually 
and  very  carefully  introduced  in  the  software.  There  are  however 
two  aspects  of  the  system  which  are  indirectly  influenced. 

First,  since  processing  is  cheaper,  there  can  be  some  tra¬ 
deoff  between  hardware  and  simplicity  of  operation.  Some  of  the 
bottlenecks  in  the  system  can  be  eased  by  more  hardware  rather 
than  smarter  algorithms  as  was  the  case  in  the  past.  The  net  ef¬ 
fect  may  be  a  system  which  is  more  robust.  Current  systems,  like 
high  perf ormance  automobiles,  are  very  sensitive.  Their  simplif¬ 
ication  may  make  them  easier  to  understand,  to  maintain  and  to 
change.  Unfortunately,  the  optimization  algorithms  are  an  in¬ 
tegral  part  of  the  systems  as  they  currently  exist.  It  will  be 
very  difficult  to  take  them  out.  We  hope  that  by  running  the 
systems  in  a  less  constrained  environment  the  optimization  stra¬ 
tegies  will  become  less  important.  If  they  are  not  used  as 
often,  they  will  eventually  migrate  to  parts  of  the  system  where 
they  can  naturally  wither  away. 

A  second  indirect  influence  on  organization  computing  ser¬ 
vices  will  be  provided  by  the  unbundling  of  personal  computing 
services.  The  net  effect  will  be  less  terminal  support  and  less 
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interrupts.  The  main  cost  of  personal  services  is  not  so  much 
actual  computing  services  provided  to  the  end  user,  hut  the  sys¬ 
tem  overhead  required  to  maintain  a  dialog  with  the  end  user. 
When  this  dialog  is  eliminated  or  reduced,  the  central  systems 
providing  organization  services  can  concentrate  more  on  large 
programs.  They  can  prohahly  run  with  lower  levels  of  multipro¬ 
gramming  and  do  task  switching  less  frequently.  At  least,  they 
will  not  have  to  deal  with  real-time  constraints  associated  with 
users  waiting  in  front  of  terminals.  As  a  matter  of  fact,  the 
on-line  operations  of  these  systems  can  he  factored  completely. 
Personal  on-line  services  can  he  provided  hy  independent  personal 
computers.  The  rest  of  the  on-line  activity  is  mainly  interac¬ 
tion  services  through  a  common  data  base.  Suppose  these  services 
are  factored  out  too  as  we  will  see  in  the  next  section.  The 
only  jobs  remaining  in  the  central  system  are  what  are  currently 
called  background  .lobs.  If  we  allow  these  jobs  to  come  into  the 
foreground  it  will  eliminate  most  of  the  bottlenecks  in  any  com¬ 
puter  center.  In  addition,  we  can  expand  the  hardware  base  to 
give  them  ample  processing  power  and  ample  real  memory.  They 
should  run  better  and  faster.  Finally,  the  computer  center  per- 
sonel  can  concentrate  on  these  important  organization  jobs  rather 
than  trying  to  answer  arbitrary  requests  and  associated  phone 
calls  from  all  sorts  of  users. 

In  summary,  cheaper  hardware  and  the  absence  of  personal 
computing  services  and  possibly  interaction  services  will  enable 
centralized  organi zation  computing  services  to  run  smoother. 
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They  will  provide  fewer  functions  to  fewer  customers  in  a  less 
constrained  environment.  They  can  afford  to  run  a  simple  opera¬ 
tion  with  a  few  well  defined  goals. 

5.  Interaction  services 

With  the  separation  of  personal  and  organization  computing 
services,  interaction  services  become  especially  important.  They 
provide  the  means  for  coordination  and  control  of  the  entire  sys¬ 
tem.  In  addition,  they  provide  the  common  link  not  only  between 
people  individually,  but  between  people  and  the  organization  as  a 
whole . 

There  are  two  types  of  interaction.  In  the  first  case, 
which  we  call  limited,  interaction,  we  are  interested  in  communi¬ 
cating  to  a  few  people  some  information  with  limited  scope  and 
short-  range  importance.  We  do  not  expect  this  information  to  be 
relevant,  or  even  understood,  by  many  people.  We  also  expect 
most  of  the  material  communicated  to  become  quickly  obsolete. 
Therefore  we  need  a  message  system  which  can  route  messages  to 
appropriate  destinations. 

In  the  second  case,  which  we  will  call  global  interaction, 
we  want  to  broadcast,  widely,  information  which  is  relevant  to 
many  and  may  have  long  range  importance.  In  this  case,  we  would 
like  to  use  a  common  data  base  which  is  accessible  by  many  people 
and  which  retains  the  information  for  further  use.  We  do  not 


claim  that  limited  interaction  can  only  be  provided  by  message 
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systems.  It  can  also  he  provided  (and  it  has  been  provided  in 
the  oast)  by  data  bases  with  appropriate  access  restrictions.  In 
addition,  global  interaction  can  be  provided,  in  principle,  with 
message  systems  which  can  broadcast  messages  widely.  Our  point 
is  that  we  may  need  both  message  systems  and  data  base  systems 
for  the  different  aspects  of  interaction. 

Both  of  these  two  kinds  of  systems  introduce  some  very  in¬ 
teresting  software  problems.  In  the  case  of  an  interaction 
oriented  data  base  service,  the  common  information  base  is  pro¬ 
vided  by  one  or  more  systems  with  a  large  'amount  of  storage. 
These  systems  are  not  file  servers,  like  the  ones  needed  in  per¬ 
sonal  computing  services.  That  is,  they  do  not  provide  just 
space  to  store  data.  Such  a  limited  service  defeats  the  goal  of 
interaction  through  a  common  information  base.  The  interaction 
oriented  data  base  systems  should  be  designed  to  process  a  large 
number  of  transactions  from  different  users  exchanging  informa¬ 
tion  through  a  common  data  base.  The  stream  of  transactions  is 
arbitrary  and  does  not  progress  in  an  orderly  fashion  through  the 
data  base.  This  situation  implies  a  rather  sophisticated  system 
which  can  handle  many  independent  transactions  in  a  real-time  en¬ 
vironment. 

It  is  not  clear  whether  these  systems  will  be  better  provid¬ 
ed  by  special  purpose  data  base  machines.  They  may  very  well  be 
provided  by  general  purpose  processors.  However,  we  expect  these 
systems  to  be  separate  from  the  systems  providing  organization 
computing  services.  Many  general  computer  systems  make  the  dis- 
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tinction  currently,  in  time,  by  running  transactions  during  the 
day  and  large  organization's  services  at  night.  In  the  future 
this  distinction  may  he  reinforced  by  providing  different  systems 
and  passing  parts  of  the  data  base  between  them.  This  type  of 
operation  will  allow  each  system  to  operate  in  a  stable  environ¬ 
ment.  We  propose,  in  fact,  that  the  batch  oriented  data  base  ac¬ 
tivity  and  the  transaction  oriented  data  base  activity  be 
separate.  This  may  seem  radical  but  the  only  link  between  the 
services  is  the  common  data  base.  This  link  can  be  preserved  by 
having  two  systems  which  have  access  to  the  same  final  repository 
of  data. 

In  the  case  of  interaction  oriented  message  systems,  there 
is  a  huge  discrepancy  between  what  a  message  system  needs  and 
what  network  protocols  provide.  High  level  protocols  for  ex¬ 
change  of  messages  freely  and  quickly  should  be  developed.  They 
need  to  ship  not  only  files  but  transactions  as  messages,  forms 
as  messages,  answers  as  messages,  letters  as  messages,  etc.  Ap¬ 
plication  oriented  message  systems  are  also  needed.  Switching  of 
messages  should  be  uniform  and  independent  of  the  desired  desti¬ 
nation  (be  that  another  personal  computer,  a  data  base  system, 
etc.).  These  systems  should  make  all  peculiarities  of  the  actual 
networks  used  transparent  (whether  they  are  broadcasting  net¬ 
works,  satellite  communications,  packet  switching,  etc.). 

We  have  discussed  limited  and  global  interaction  separately 
as  being  independent  of  each  other.  As  a  matter  of  fact,  they 
are  not.  There  is  a  natural  transition  where  some  of  the  infor- 
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mation  that  is  treated  as  a  limited  interaction  "becomes  globally 
relevant  and  important  and  should  be  treated  as  a  global  interac¬ 
tion.  If  message  systems  offer  limited  interaction  and  data  base 
systems  offer  global  interaction,  then  there  should  be  a  natural 
connection  among  them.  They  should  not  be  developed  in  isola¬ 
tion.  Information  should  be  able  to  migrate  between  message  sys¬ 
tems  and  data  base  systems  easily.  The  migration  of  information 
implies  some  common  models  and  maybe  complicated  translation  al¬ 
gorithms  . 

The  second  connection  is  between  global  interaction  services 
and  organization  computing  services.  The  organization  services 
operate  on  data  provided  by  the  global  interaction  services. 
This  was  the  case  in  the  past  where  data  base  systems  had  a  batch 
operation  aspect  and  an  on-line  transaction  aspect.  This 
cooperation  should  be  preserved  even  if  the  data  base  transaction 
aspects  are  separated  from  the  organization  computing  services. 

6.  Concluding  remarks 

Technology  is  changing  fast.  Maybe  the  shape  of  components 
is  the  same  but  their  cost  is  quite  different.  This  situation 
influences  very  much  the  overall  system  architecture,  function 
and  utilization  of  computers.  We  expect  computer  systems  to 
evolve  in  the  following  directions. 

1)  Personal  computers  with  clusters  of  peripheral  servers  provid¬ 
ing  personal  computing  services  with  emphasis  on  ease  of  use. 
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2)  Large  centralized  systems  running  in  a  simplified  manner  to 
provide  organization  computing  services. 

3)  Message  systems  providing  an  environment  for  limited  interac¬ 
tion. 

4)  Transaction  oriented  data  base  systems  providing  global  in¬ 
teraction  with  connection  to  both  organization  services  and  mes¬ 
sage  systems. 

If  we  accept  this  view  there  are  some  major  changes  in  terms 
of  software  directions. 

1)  There  is  a  need  for  end  user  tools  in  personal  computing  ser¬ 
vices  as  opposed  to  systems  tools,  e.g.,  systems  programming 
languages . 

2)  Some  traditional  areas  of  software  engineering,  e.g.,  mul¬ 
tiprogramming,  dynamic  resource  allocation,  scheduling,  process 
switching,  etc.  become  less  important  since  we  mainly  operate  in 
a  uniprogramming  environment  or  close  to  it. 

3)  Transaction  oriented  data  base  systems  become  extremely  impor¬ 
tant  as  a  common  link  between  personal,  organization  and  interac¬ 
tion  services. 

4)  Message  systems  should  be  developed  which  provide  a  uniform 
interaction  tool  irrespective  of  network  characteristics  and 
which  can  interface  readily  with  data  base  systems. 
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The  major  shift,  in  our  opinion,  can  he  summarized  by  the 
motto  "Trade  hardware  for  simplicity".  For  a  long  time,  we  have 
been  trying  to  develop  techniques  and  methodologies  to  deal  with 
complexity.  We  had  partial  success  in  doing  so.  However,  the 
application  of  all  these  techniques  is  not  that  easy.  Maybe  we 
should  acknowledge  that  computer  scientists  are  no  more  intelli¬ 
gent  and  creative  than  other  scientists.  Rather  than  always  try¬ 
ing  to  deal  with  complexity,  we  should  try  to  remove  some  of  it. 
Hardware  is  getting  cheap.  We  can  sacrifice  some  of  this  good 
fortune  to  make  our  systems  simpler  and  our  lives  easier. 
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Implementing  a  Microcomputer  Database  Management  System 
1.  Introduction 

MPS  is  a  standalone  microcomputer  database  system  with  a 
powerful  interactive  query  language  based  on  a  subset  of  SQL 
[CHAM76] .  SQL  is  a  relational  query  language  developed  at  IBM 
for  system  R,  a  large  relational  database  system  designed  for 
the  370  series  of  computers.  The  microcomputer  database  is 
optimized  to  efficiently  handle  databases  up  to  two  megabytes 
in  size.  The  entire  database  resides  on  inexpensive  floppy  disks 
with  each  floppy  disk  handling  roughly  one  megabyte  of  online 
data . 


One  outstanding  feature  of  the  microcomputer  database  system 
is  low  cost.  The  hardware  for  the  entire  system  can  be  purchased 
for  roughly  $7,000  U.S.  [LYCK77] .  This  cost  includes  a  microcom¬ 
puter,  sufficient  memory,  floppy  disks  and  a  video  terminal. 
However,  the  cost  could  go  as  high  as  $15,000  if  the  user  desires 
extra  disks  and  wants  to  purchase  the  system  pre=-assembled .  The 
low  cost  of  the  database  system  opens  up  application  areas  where 
prohibitive  costs  have  precluded  the  use  of  computerized  sys¬ 
tems.  The  low  initial  cost  of  the  system  also  allows  the 
development  of  an  interesting  and  useful  distributed  configura¬ 
tion  without  the  necessity  of  an  enormous  budget.  In  fact,  the 
flexibility  of  microcomputers  allows  several  potential  configura¬ 


tions  to  be  built  and  evaluated. 
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2.  Selection  of  Microcomputer  Hardware  and  Software. 

Many  microprocessor  systems  were  examined.  In  general*  most 
systems  belonged  to  one  of  two  categories.  The  first  category 
consisted  of  the  new,  faster  microprocessor  systems.  It  was 
found  that  the  operating  systems  available  for  these  machines 
were  often  in  the  development  phase  and  not  fully  tested.  The 
second  category  consisted  largely  of  the  established  microproces¬ 
sor  systems.  It  was  found  that  the  operating  systems  for  these 
machines  were  very  restrictive.  In  almost  all  cases,  there  was  a 
lack  of  a  suitable  systems  implementation  language. 

Only  one  microcomputer  candidate,  the  Digital  Equipment 
Corporation's  LSI-11  microconputer  system,  running  the  UNIX 
operating  system,  had  the  desired  characteristics  required  for  a 
microcomputer  database  system  implementation.  Although  the  LSI- 
11  lacks  memory  management  hardware  and  is  restricted  to  a  max¬ 
imum  memory  size  of  28K  words  of  main  memory,  its  16-bit  word 
length  and  fast  execution  speed  provides  enough  power  to  build  a 
database  system.  Because  of  economic  constraints,  floppy  disks 
were  chosen  as  the  storage  medium  for  the  database.  This  res¬ 
tricts  the  size  of  the  database  to  roughly  two  megabytes. 

The  UNIX  operating  system  supports  the  C  programming 
language.  UNIX  has  useful,  well-designed  file  system  primitives 
and  in  general,  provides  quick  response  time.  The  C  programming 
language  is  a  block  structured  language  having  features  similar 
to  ALGOL,  PASCAL,  and  PL/1. 
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One  serious  problem  remained:  the  UNIX  operating  system  is 
designed  to  run  only  on  the  larger  PDP-11  computers  having  suit¬ 
able  memory  management  hardware.  A  major  redesign  of  the  operat¬ 
ing  system  would  be  required  in  order  to  make  it  run  on  the  LSI- 
11  microcomputer.  However,  Bell  Labs  has  another  software  pro¬ 
duct,  MINI-UNIX,  which  is  similar  to  UNIX  but  designed  for  a 
small  PDP-11  like  the  PDP-11/05  minicomputer  which  is  not  unlike 
the  LSI-11.  Therefore  MINI-UNIX  was  then  adapted  to  run  on  the 
LSI-11  microcomputer. 

3.  Data  Model 

MRS  expects  to  work  in  a  real  environment  in  which  it  will 
be  exposed  to  users  of  varying  sophistication,  ranging  from 
secretaries  to  computer  scientists.  For  this  reason,  it  is 
desirable  to  consider  a  high  level  user  interface  and  minimize 
the  internal  data  management  primitives.  A  high-level  interface 
simplifies  operations  for  the  user  and  reduces  the  length  of 
training  required.  At  the  internal  level,  it  is  desirable  to 
present  a  small  set  of  database  management  primitives  which  are 
easily  implementable  in  terms  of  the  file  primitives  provided  by 
the  operating  system. 

Each  of  the  hierarchical,  network  and  relational  data  models 
has  suitable  high-level  query  languages  and  database  management 
primitives.  The  key  to  choosing  among  these  data  models  then,  is 
the  complexity  of  the  interface  between  the  external  and  internal 
levels,  and  the  difficulty  in  translating  the  high-level  query 
language  operations  to  the  low-level  database  management  primi- 
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tives.  A  study  of  the  three  data  models  suggests  that  the  use  of 
the  relational  data  model  at  both  levels  will  result  in  the 
greatest  simplicity.  The  mark  of  a  high-level  query  language  is 
its  emphasis  on  operations  on  sets  of  records  as  opposed  to  indi¬ 
vidual  records.  Whereas  the  database  management  primitives  of 
the  relational  data  model  are  typically  set-oriented,  the  data¬ 
base  management  primitives  of  the  hierarchical  and  network  data 
models  are  typically  navigational.  None  of  the  sets  of  database 
management  primitives  present  much  difficulty  in  implementation, 
but  the  translation  of  the  set  operations  of  the  query  language 
to  navigational  primitives  is  considerably  more  difficult  than 
that  to  set-oriented  primitives. 

4 .  Query  Language 

Of  the  many  high-level  relational  query  languages  proposed 
and  implemented,  one  of  the  most  widely  known  is  SQL  [CHAM76] . 
SQL  is  based  on  English  keywords  and  is  intended  to  serve  a  broad 
spectrum  of  computer  users.  The  MRS  query  language  SLQ  is  derived 
from  SQL.  The  SLQ  command  repertoire  provides  for  the  defining, 
retrieving,  inserting,  deleting  and  updating  of  data.  This  por¬ 
tion  of  SLQ  is  essentially  downward-compatible  with  SQL.  Addi¬ 
tional  SLQ  commands  are  used  for  the  display  and  control  of  the 
UNIX-based  MRS  environment. 

A  database  of  Barbados  fish  specimens  was  obtained  from  the 
Royal  Ontario  Museum,  Toronto,  Canada.  It  is  used  to  illustrate 
various  features  of  the  query  language.  The  example  database  is 
composed  of  the  following  two  relations: 
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specimen  (family,  genus,  species,  length,  no-specimens, 
day,  month,  year,  hours,  locale-no)  and 

locale  (locale-id,  county,  location,  waterbody, 
depth-range,  vegetation,  bottom). 

The  specimen  relation  describes  the  characteristics  of  the  fish: 
its  family,  genus  and  species,  its  length  in  centimeters,  the 
number  caught,  and  the  date  and  duration  of  the  catch  session. 
The  locale  relation  describes  the  physical  environment  of  the 
catch  site:  the  county,  location  and  water  body,  the  depth-range, 
the  subsurface  vegetation,  and  the  composition  of  the  ocean 
floor . 

The  SLQ  syntax  is  free  format.  Planks,  ends  of  lines  and 
comments  delimited  by  are  ignored.  signifies  the  end  of  a 
command.  Keywords  may  be  either  entirely  lowercase  or  uppercase. 
Here,  they  are  shown  in  uppercase  for  emphasis. 

The  CREATE  TAELE  command  defines  new  relations  in  the  data¬ 
base.  The  user  supplies  the  relation  name,  the  attribute  names 
and  their  associated  types.  The  types  are  currently  either  char¬ 
acter  or  numeric.  The  character  type  allows  strings  of  varying 
length  up  to  the  maximum  specified.  The  numeric  type  allows  both 
positive  and  negative  integers.  Additional  types  are  planned. 
The  following  command  creates  the  locale  relation. 
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CREATE  TABLE  locale 


loca le-id 

NUMERIC , 

county 

CHAR 

(15)  , 

location 

CHAR 

(50)  , 

waterbody 

CHAR 

(20)  , 

depth-range 

CHAR 

(9), 

vegetation 

CHAR 

(15)  , 

bottom 

CHAR 

(15))! 

The  DROP  TABLE  command  removes  relations  from  the  database. 
The  following  command  removes  the  locale  relation. 

DROP  TABLE  locale? 

The  CREATE  INDEX  command  creates  named  indices  on  attributes 
of  relations.  The  following  command  creates  an  index  named 
genus-index  on  the  genus  attribute  of  the  specimen  relation. 

CREATE  INDEX  genus-index  ON  specimen  (genus); 

The  DROP  INDEX  command  removes  indices. 

The  following  command  removes  the  index  genus-index. 

DROP  INDEX  genus-index? 

The  SELECT  command  performs  retrievals.  It  contains  a 
SELECT-ITEM  list,  a  FROM  list  and  an  optional  WHERE  clause.  The 
SELECT-ITEM  list  specifies  the  attributes  to  be  retrieved,  the 
FROM  list  the  relations  involved,  and  the  WHERE  clause  the  cri¬ 
teria  of  selection.  The  following  command  lists  all  the  dif¬ 


ferent  water  bodies  and  their  subsurface  vegetation 
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SELECT  waterbody,  vegetation 
FROM  locale; 

If  the  SELECT-ITEM  list  (i.e.  waterbody,  vegetation)  is  omitted, 
all  attributes  are  retrieved.  The  following  command  lists  the 
specimens  (family,  genus,  species)  belonging  to  the  family  POTHI- 
DAE  or  POMACENTRIDAE  which  were  caught  before  1964. 

SELECT  family,  genus,  species 
FROM  specimen 
WHERE  year  <  1964 

AND  (family  =  'BOTHIDAE' 

OR  family  =  'POMACENTRIDAE'); 

The  WHERE  clause  consists  of  AND  and  OR  connectives  and 
predicates  for  comparing  attributes  with  other  attributes  or  con¬ 
stants.  Parentheses  are  used  to  establish  the  precedence  of  the 
connectives.  The  following  command  counts  the  number  of  catch 
sites  with  the  ocean  floor  not  composed  of  sand/coral. 

SELECT  COUNT 

FROM  locale 

WHERE  bottom  NOT  =  'SAND/CORAL ' J 

Statistical  options:  MAX,  MIN,  SUM  and  AVG  are  also  avail¬ 
able  for  numeric  attributes.  The  following  command  finds  all 
genera  caught  in  the  same  locale  as  the  specimen  MUGIL  CUREMA. 
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SELECT  genus 

FROM  specimen 
WHERE  locale-no  IN 

SELECT  locale-no 

FROM  specimen 
WHERE  genus  =  'MUGIL' 

AND  species  =  'CUREMA'J 

The  preceding  query  illustrates  a  nested  SELECT.  The  inner¬ 
most  SELECT  is  processed  first  producing  a  collection  of  single 
attribute  values  which  is  then  used  as  a  selection  criterion  of 
the  outer  SELECT.  A  SELECT  may  be  nested  as  many  times  as 
desired.  The  following  command  lists  all  the  species,  the  number 
caught  and  the  depth  range  of  the  catch  site  if  the  genus  is 
BOTHUS. 

SELECT  species,  no-specimens,  depth-range 
FROM  specimen,  locale 
WHERE  locale-no  =  locale-id 

AND  genus  =  'BOTHUS'; 

Joins  are  formed  when  more  than  one  relation  is  present  in  the 
FROM  list.  The  WKFRE  clause  specifies  the  .loin  condition. 

i 

The  INSERT  command  inserts  a  tuple  into  a  relation.  The 
following  command  inserts  a  tuple  into  the  locale  relation. 
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INSERT  INTO  locale: 

[300, 

'CHRIST  CHURCH', 

'180M  OFF  SHORE  NEAR  OISTINS  TOWN  ON  S  COAST', 

NULL, 

'1.6-2. 4', 

'THALASSI A ' , 

'RUEBLE/SAND']  J 

The  DELETE  command  deletes  tuples  from  a  relation.  Its 
WHERE  clause  selects  the  appropriate  tuples.  The  following  com¬ 
mand  deletes  all  tuples  of  family  BOTHIDAE. 

DELETE  specimen 

WHERE  family  =  'BOTHIDAE'; 

The  UPDATE  command  updates  attribute  values  in  tuples  of  a 
relation.  Its  WHERE  clause  selects  the  appropriate  tuples.  The 
following  command  sets  the  locale-no  to  400  for  all  specimens 
with  locale-no  =  300. 

UPDATE  specimens 

SET  locale-no  =  400 
WHERE  locale-no  =  300; 

The  DISPLAY  command  displays  the  attributes  of  a  relation. 
The  following  command  displays  the  locale  relation. 

DISPLAY  TABLE  locale; 


The  output  is: 
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locale-id 


NUMERIC 


c  ounty 


CHAR  (15) 


1 ocat ion 


CHAR  (50) 


waterbody 


CHAR  (20) 


depth-range 


CHAR  (9) 


vegetation 


CHAR  (15) 


bottom 


CHAR  (15) 


5.  Database  Structures 

The  MRS  database  is  implemented  on  top  of  the  UNIX  file  sys¬ 
tem.  The  UNIX  file  system  provides  many  useful  features.  First, 
being  hierarchically  structured,  the  file  system  permits  files  to 
be  grouped  under  directories  and  sub-directories.  Thus,  each  MRS 
database  comprises  a  group  of  files  under  one  directory  and  many 
MRS  databases  can  exist  in  one  file  system  under  separate  direc¬ 
tories.  Second,  the  structure  of  data  in  a  file  is  left  com¬ 
pletely  undefined  by  UNIX.  To  allow  full  exploitation  of  this 
policy,  UNIX  provides  a  complete  set  of  file  primitives  for  the 
reading  and  writing  of  any  character  or  sequence  of  characters 
anywhere  in  a  file.  MRS  can  thus  impose  its  own  structure  on  the 
data  in  a  file.  Third,  UNIX  provides  multiple  buffering  for  file 
input/output  operations  on  a  least  recently  used  (LRU)  replace¬ 
ment  basis.  Thus,  random  file  operations  are  not  necessarily 
inefficient  and  MRS  does  not  have  to  do  its  own  data  buffering. 

Each  MRS  database  comprises  a  data  directory,  a  number  of 
relations  and  a  number  of  indices.  See  figure  1  for  an  example. 
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The  data  directory  spans  two  files,  one  containing  information  on 
the  relations  in  the  database  and  the  other  containing  informa¬ 
tion  on  the  attributes  of  each  relation.  The  information  is 
stored  as  contiguous  fixed-length  records  in  the  files.  The 
fields  in  a  relation's  record  contain  the  name  of  the  relation, 
the  length  of  a  tuple  of  the  relation,  the  number  of  attributes, 
and  a  pointer  to  the  corresponding  records  in  the  attributes 
file.  The  fields  in  an  attribute's  record  contain  the  name  of 
the  attribute,  the  length  of  the  attribute  value  ir.  a  tuple,  the 
position  of  the  value,  the  type,  whether  character  or  numeric, 
and  the  name  of  an  index  on  the  attribute,  if  any. 

Each  relation  in  the  database  occupies  a  single  data  file. 
The  name  of  the  file  is  the  name  of  the  relation.  It  must  be 
unique  within  the  database.  The  tuples  of  the  relation  are 
stored  as  contiguous  fixed-length  records  in  the  file.  The 
fields  of  the  records  correspond  to  the  attributes  of  the  rela¬ 
tion. 

Each  index  on  an  attribute  of  a  relation  consists  of  two 
files,  one  containing  the  different  values  for  the  attribute  and 
the  other  containing  pointers  to  tuple  records  of  the  relation. 
See  figure  2  for  an  example.  The  values  file  contains  a  standard 
E-tree  structure  for  fast  searching.  Each  E-tree  block 
corresponds  to  a  logical  block  of  the  file  system  to  minimize 
actual  file  reads.  Each  E-tree  record  contains  an  attribute 
value,  one  pointer  to  a  tuple  record  of  the  relation  and  one 
pointer  to  a  record  in  the  pointers  file.  The  idea  behind  this 
scheme  is  that  one  single  format  of  indexing  suffices  for  both 
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primary  and  secondary  indices.  If  the  index  is  primary,  one 
pointer  to  a  tuple  record  is  adequate  and  the  pointers  file  is 
unused.  If  the  index  is  secondary,  many  pointers  to  tuple  records 
are  needed  and  the  pointers  file  is  necessary.  The  pointers  file 
contains  fixed-length  records  of  pointers  to  tuple  records. 
Records  for  pointers  corresponding  to  the  same  attribute  value 
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are  linked  together  in  a  list. 


*<~ 

*<~ 

*<~ 


* 


B 

C 

D 


/■ 

i 

+ 


V 


>■ 


> 


- \ 

1  \ 

— \  \— > 


\ 


\ 


\ 


\-> 


B 

E 

E 

C 

D 

D 


< - 


< 


■\ 

i 

i 

t 


\ 

•  1 

• 

• 

!  !  •  1 

•  i  •  •  •  i  • 

.  i  i  T  i 

/— ■ 

/ 

. .  i  i  T  i 

•  j  •  •  •  1  A  | 

•  | 

•  | 

• 

<-/ 

— \ 
i 
i 

<-/ 

<-\ 


7 


A  E-tree  Elock 
in  Values  File 
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Figure  2 

The  database  structures  are  simple.  The  structures  of  the 
data  directory  and  the  relations  are  identical.  As  fixed-length 
records  are  used,  operations  such  as  random  access,  reorganiza¬ 
tion,  data  compaction  and  garbage  collection  are  easy  to  imple¬ 
ment.  The  structure  of  the  relations  is  also  entirely  indepen¬ 
dent  of  the  structure  of  the  indices.  This  permits  the  structure 
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of  either  to  change  with  minimal  impact  on  the  other.  In  fact, 
in  the  course  of  development,  MRS  changed  the  indexing  structure 
from  ISAM  to  B-trees  with  no  effect  on  the  rest  of  the  database 
structures.  Furthermore,  this  independence  allows  relations  to 
exist  without  indices. 

6.  The  Use  of  Indices 

The  proper  use  of  indices  is  an  important  issue.  While 
indices  are  necessary  for  efficient  access  to  relation  tuple 
records,  the  data  storage  occupied  by  the  index  structures  is 
often  substantial.  Hence,  it  is  important  to  maintain  only  the 
necessary  index  structures  and  to  make  use  of  them  whenever  pos¬ 
sible.  Since  the  automatic  selection  of  indicies  by  the  database 
system  is  a  difficult  and  as  yet  unsolved  problem  [KING74, 
SCHK75a,  SCHK75b] ,  the  MRS  user  has  full  control  over  the 
indices.  He  may  create  or  destroy  them  as  he  wishes. 

MRS  does  not  require  a  primary  index  on  the  key  attribute  of 
a  relation.  Consequently,  MRS  also  does  not  support  the  key  con¬ 
cept  directly,  for  the  maintenance  of  the  key  can  only  be  done 
efficiently  with  a  primary  index.  The  justification  for  this 
non-support  of  the  key  lies  in  the  fact  that  a  primary  index  for 
the  key  may  not  be  cost-effective  for  retrieval  purposes.  Con¬ 
sider  two  cases.  The  first  case  concerns  the  relation: 

object  (catalog#,  name,  price,  ...) 
with  catalog#  as  its  key.  If  access  to  this  relation  were  mostly 
by  name  and  infrequently  by  catalog#,  then  a  secondary  index  on 
name  would  be  sensible  and  the  primary  index  on  catalog#  would 


Implementing  a  Microcomputer  Database  Management  System 


36 


be,  in  most  cases,  superfluous.  A  system  based  on  microcomput¬ 
ers  and  with  limited  secondary  storage  capacity  can  ill  afford 
superfluous  structures.  However,  certain  applications  may  still 
require  the  primary  index  in  order  to  enforce  certain  con¬ 
straints.  The  second  case  concerns  two  relations: 

employee  (name,  address,  child ren-code ,  ...)  and 
child  (parent-code,  first-name,  age,  ...) 
with  name  and  <pa re nt -code ,  first-name>  as  their  respective  keys. 
If  the  children  relation  will  not  be  accessed  on  its  own,  but 
only  in  a  join  with  the  employee  relation  under  the  condition: 

i 

empl oyee . chi ldren-code  =  child . parent-code , 
then  primary  indices  on  <parent-code ,  first-name>  would  be 
wasted.  They  would  not  facilitate  the  join.  In  view  of  this,  a 
compromise  for  MRS  is  to  support  the  key  concept  in  those  cases 
where  the  user  decides  to  create  a  primary  index  for  the  key. 

In  accordance  with  the  user  control  of  indices,  the  MRS 
database  access  policy  is  to  make  use  of  indices  whenever  possi¬ 
ble  and  to  do  sequential  scans  on  the  relations  otherwise.  The 
intelligent  use  of  existing  indices  depends  entirely  on  the  map¬ 
ping  from  the  high-level  query  language  operations  to  the  low- 
level  database  management  primitives.  This  is  a  difficult  optim¬ 
ization  problem  [ELAS77,  W0NG76] ,  and  MRS  relies  on  obvious 
heuristics  to  handle  the  problem. 

The  need  for  indices  arises  in  the  selection  of  relation 
tuples.  In  SLQ ,  selection  is  specified  by  the  WHERE  clause  con¬ 
sisting  of  conjunctions  and  disjunctions  of  predicates.  It  is 
well-known  that  disjunctions  are  problematical.  Hence,  MRS 
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examines  only  predicates  visible  with  conjunctions  at  the  top 
level.  For  example,  in  the  following  WHERE  clause: 

WHERE  A  AND  ( R  OR  (C  AND  D))  AND  (E  AND  F) 
only  A,  E  and  F  are  examined.  E,  C  and  D  are  hidden  under  a 
level  of  disjunction.  Two  forms  of  examined  predicates  are  use¬ 
ful. 

The  first  form  compares  an  indexed  attribute  to  a  constant 
value.  Pointers  to  tuple  records  which  satisfy  the  predicate  are 
quickly  found  through  the  index  and  kept  in  a  list.  When  all  use¬ 
ful  predicates  have  been  used  to  compile  lists,  an  intersection 
of  the  lists  is  performed  in  accordance  with  the  conjunctions. 
The  resulting  list  gives  a  smaller  number  of  tuple  records  to  be 
checked  against  the  remainder  of  the  WHERE  clause. 

The  second  form  is  an  implicit  join  comparing  an  indexed 
attribute  of  one  relation  to  an  attribute  of  another  relation. 
As  before,  useful  predicates  of  the  first  form  are  used  to  reduce 
the  number  of  candidate  tuple  records  of  the  two  relations  for 
the  potential  join  cross-product.  Then,  for  each  candidate  record 
of  the  second  relation,  the  value  of  its  join  attribute  is 
extracted  and  used  for  scanning  the  index  of  the  join  attribute 
of  the  first  relation.  The  list  of  records  obtained  from  this 
index  and  the  list  of  candidate  records  previously  obtained  are 
intersected  to  give  a  smaller  number  of  records  of  the  first 
relation.  These  records  are  then  joined  to  the  corresponding 
record  of  the  second  relation  and  finally  checked  against  the 
remainder  of  the  WHERE  clause. 
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The  described  heuristics  are  not  optimal.  However  for  sim¬ 
ple  tuple  selection,  they  are  adequate.  The  algorithms  are  sim¬ 
ple  and  their  implementation  does  not  entail  very  lengthy  code. 
This  is  important  in  view  of  the  limitations  of  the  microcom¬ 
puter.  In  the  future  where  these  limitations  no  longer  apply  and 
the  efficient  use  of  indices  is  better  understood,  MBS  expects  to 
employ  better  algorithms. 

7.  Software  Structures 

The  database  management  software  of  MRS  is  extremely  modu¬ 
lar.  The  query  language  program  takes  advantage  of  the  UNIX 
multi-processes  capability  by  being  split  into  a  number  of 
processes  which  invoke  each  other  in  a  set  sequence.  The  initial 
process  invoked  by  the  user  is  the  driver  process.  The  user 
input  interface  is  isolated  in  the  input  process.  Each  SLQ  com¬ 
mand  is  performed  by  a  separate  function  process.  Their  rela¬ 
tionship  is  summed  up  by  the  following  diagram: 
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The  typical  sequence  of  events  for  each  SLO  command  is  as 
follows.  First,  the  driver  process  passes  control  to  the  input 
process.  After  the  user  enters  his  command,  the  input  process 
parses  it  and  constructs  a  parse  tree  as  internal  representation. 
Then  it  sends  the  parse  tree  to  the  driver  process  and  returns 
control.  The  driver  process  determines  the  appropriate  function 
process  from  the  parse  tree,  sends  the  parse  tree  to  the  process 
and  passes  control.  After  the  function  process  interprets  the 
parse  tree  and  performs  the  required  operation,  it  returns  con¬ 
trol  to  the  driver  process.  At  this  point,  a  new  command  cycle 
starts.  This  process  is  terminated  when  a  STOP  command  is 
issued . 

The  parser  of  the  input  process  is  written  in  YACC  [J0HN75] 
--  Yet  Another  Compiler  Compiler.  YACC  takes  for  input  a  large 
class  (LALR  (1))  of  input  grammars  expressed  in  an  enhanced  ENE- 
like  notation  and  resolves  ambiguities  using  precedence  informa¬ 
tion.  The  use  of  YACC  allows  the  SLO  language  syntax  to  be 
implemented  cleanly  and  quickly.  The  parse  tree  generated  in 
parsing  is  an  applicative  expressive  (AE)  tree  [LAND64] .  It 
differs  from  common  parse  trees  in  that  each  tree  node  represents 
an  action  to  be  performed  rather  than  a  production  in  the  course 
of  parsing.  It  is  especially  suitable  for  the  direct  translation 
of  nodes  into  operations  during  tree  traversal.  It  is  also  quite 
compact  so  that  the  overhead  in  sending  it  from  process  to  pro¬ 
cess  is  small. 

Each  of  the  function  processes  comprises  three  parts:  a  com¬ 
mon  module  that  accepts  the  parse  tree  from  the  driver  process 
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and  sets  it  up  in  an  internal  workspace,  an  interpreter  that 
traverses  the  parse  tree  and  translates  it  into  database  manage¬ 
ment  primitives,  and  the  set  of  database  management  primitives 
that  use  the  UNIX  file  primitives  to  accomplish  their  task.  The 
set  of  primitives  forms  the  basis  of  a  host  language  interface. 

At  this  point,  the  actual  mechanism  for  interprocess  commun¬ 
ications  must  be  considered.  UNIX  provides  an  interprocess  com¬ 
munications  device  called  a  pipe.  A  pipe  is  essentially  a  uni¬ 
directional  channel  implemented  with  memory  resident  buffers.  As 
only  memory  transfers  are  involved,  the  overhead  in  transmitting 
a  parse  tree  through  a  pipe  is  small.  Unfortunately,  MINI-UNIX 
does  not  support  pipes.  Hence,  under  MINI-UNIX,  the  alternative 
for  MRS  is  to  employ  regular  UNIX  files  for  communications. 
After  the  input  process  constructs  the  parse  tree,  the  parse  tree 
is  copied  into  a  designated  file.  The  driver  and  function 
processes  may  subsequently  read  this  file.  The  use  of  files  for 
communications  slows  the  system  considerably  due  to  the  required 
accesses  to  the  secondary  storage  device.  However,  that  cannot 
be  avoided  without  major  modifications  to  the  MINI-UNIX  operating 
sys  tern. 

8.  Testing  and  Performance. 

Most  of  the  development  work  for  MRS  was  carried  out  on  a 
large  UNIX  installation.  As  a  result,  once  development  had  been 
completed,  it  was  possible  to  execute  the  system  on  various  PDP- 
11  configurations.  Development  on  a  larger  machine  gave  the 
system  an  added  flexibility  not  found  in  most  microcomputer  sys- 
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terns.  If  a  user  who  is  running  the  system  on  a  small  machine 
decides  that  he  needs  more  computer  power,  he  need  only  buy  a 
larger  PDP-11  computer.  The  MRS  software  has  been  tested  on  all 
models  of  the  PDP-11  family  that  are  capable  of  running  either 
UNIX  or  MINI-UNIX.  Most  microprocessor  systems  require  a  com¬ 
plete  redesign  before  they  can  be  moved  to  another  hardware  con¬ 
figuration  . 

It  was  felt  that  by  testing  the  system  on  several  configura¬ 
tions  any  limitiations  imposed  by  the  microcomputer  hardware 
would  surface.  The  performance  evaluation  consisted  of  five  dif¬ 
ferent  hardware  configurations  ranging  from  the  LSI-11  microcom¬ 
puter  to  the  top-of-the-1 i ne  PDP-11/70.  In  addition  to  testing 
on  different  CPU  configurations,  several  varieties  of  disks  were 
also  evaluated.  These  ranged  from  the  slow  but  inexpensive 
floppy  disks  to  the  high  capacity,  high  performance  3330-type 
disk  which  is  a  standard  secondary  storage  device  seen  often  in 
large  installations. 

Unfortunately,  it  was  not  feasable  to  check  all  combinations 
of  disks  on  all  combinations  of  CPU's.  However,  a  representative 
sampling  was  taken  and  fortunately  this  showed  several  clear  cut 
trends.  It  should  be  noted  that  the  first  three  configurations 
were  running  a  modified  version  of  MINI-UNIX  while  the  remaining 
configurations  ran  UNIX. 

In  each  case,  the  Parbados  fish  database  was  used  to  conduct 
the  performance  tests.  The  fish  database  had  indicies  built  for 
the  attributes:  GENUS  and  LOCALE. ID.  The  SPECIMENS  and  LOCALE 
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relations  were  1400  tuples  x  118  bytes  and  214  tuples  x  136  bytes 
respectively  in  length.  Queries  one  through  five  retrieved  214, 
65,  210,  45,  and  11  tuples  respectively. 

Each  test  measured  the  elapsed  time  in  seconds  required  to 
process  a  given  query.  In  each  case  the  tests  were  conducted  on 
a  standalone  system  with  no  other  users.  It  was  felt  that  the 
presence  of  other  users,  although  realistic  of  an  actual  operat¬ 
ing  environment,  would  complicate  the  results.  Also,  because  of 
the  slow  swapping  speed  associated  with  the  floppy  disks  it  is 
not  realistically  possible  to  run  a  multi-user  system  using  these 
disks . 

For  each  of  the  different  hardware  configurations  five  exam¬ 
ple  queries  formed  the  basis  of  the  test.  In  addition  to  these 
queries,  two  additional  queries  were  executed.  Each  of  these 
queries  were  essentially  linear  scans  through  both  of  the  rela¬ 
tions  in  the  database.  The  following  three  tables  outline  the 
different  hardware  configurations  used  for  the  test,  the  queries, 
and  the  time  required  to  process  the  queries. 
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Hardware  Configuration  for  Performance  Evaluations 


Configuration 

CPU 

Swapping 

Disk 

Database 

Disk 

Terminal 

Speed 

1 

LSI-11 

AED-f 1 oppy 

AED-f loppy 

1200  Baud 

2 

11/34 

AED-f loppy 

AED-f loppy 

1200  Baud 

3 

11/34 

Hard  Disk 

AED-f loppy 

1200  Baud 

4 

11/45 

Hard  Disk 

Hard  Disk 

9600  Baud 

5 

11/70 

Hard  Disk 

Hard  Disk 

1200  Eaud 

Queries  used  to  benchmark  the  database. 


1)  Select  unique  waterbody,  vegetation 

from  locale. 

2)  Select  family,  genus  specimens 

from  specimens 
where  year  <  1964 

and  (family  =  'BOTHIDAE'  or 
family  =  'POMACENTRI DAE  '  ) J 


3) 

Select 

count 

from  1 

ocale 

where 

bottom 

not  = 

'SAND/CORAL'; 

4) 

Select 

unique 

genus 

from  s 

pecime 

ns 

where 

locale 

jo  in 

select 

local 

e_no  from 

specimens 

where 

genus  = 

MUGIL'  and 

species  = 

'CUREMA'J 

5) 

Select 

specie 

s  ,  no_ 

specim 

ens,  depth 

^range 

from  s 

pecime 

ns ,  lo 

cale 

where 

locale 

_no  = 

locale  id 

and  no 

_speci 

mens  >  10» 

I rrplemen ting  a  Microcompute 

MRS  Pe 


1 


SPECIMENS 

81 

LOCALE 

31 

Query 

1 

78 

Query 

2 

108 

Query 

3 

33 

Query 

4 

88 

Query 

5 
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(note  that 
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formance  Measurements 


Configurations 


2 

3 

4 

5 

77 

36 

10 

6 

31 

9 

3 

3 

77 

76 

22 

14 

92 

62 

25 

14 

31 

10 

7 

3 

82 

39 

27 

12 

39 

16 

16 

4 

the  times  are  in  seconds) 


The  performance  measurements  reveal  some  interesting  trends. 
First,  the  performance  of  the  system  appears  to  he  adequete  even 
on  the  slowest  hardware.  By  looking  at  the  measurements  it  is 
possible  to  determine,  to  an  extent,  which  resources  are  critical 
to  the  overall  performance  of  the  system.  By  looking  at  the  tim¬ 
ings  for  the  first  and  second  configurations  it  can  be  observed 
that  there  is  practically  no  difference  in  the  elapsed  times. 
However,  the  CPU  in  configuration  two  runs  twice  as  fast  as  the 
CPU  in  configuration  one.  One  can  conclude  that  for  a  system 
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using  only  floppy  disks,  the  CPU  speed  is  not  especially  impor¬ 
tant.  It  is  clear  that  in  both  configurations  one  and  two,  the 
systems  are  heavily  I/O  bound. 

Configurations  two  and  three  differ  in  the  device  used  for 
swapping.  Configuration  two  uses  a  floppy  disk  for  swapping, 
while  configuration  three  uses  a  much  faster  hard  disk  for  swap¬ 
ping.  The  timing  of  queries  in  these  two  columns  vary.  For 
tasks  that  require  little  process  switching,  the  timings  are 
essentially  the  same.  However,  those  queries  that  required  a 
significant  amount  of  process  switching  greatly  benefited  from 
the  fast  swap  device.  The  timings  for  these  queries  were  roughly 
one  half  of  the  queries  that  required  little  process  switching. 

The  timings  obtained  for  configurations  four  and  five  show 
what  performance  can  be  had  with  fast  hardware.  It  is  clear  that 
fast  disks  present  on  fast  machines  can  make  up  to  an  order  of 
magnitude  difference  in  the  speed  of  some  queries. 

It  is  clear  from  the  timing  estimates  that  an  LSI-11  confi¬ 
guration  would  substantially  benefit  from  a  faster  disk  drive. 
Although  such  a  configuration  was  not  available  for  testing,  by 
comparing  the  timings  for  configuration  two  and  four  (the  CPU 
speeds  are  roughly  comparable)  it  is  shown  that  fast  disks  make 
an  appreciable  difference  in  performance. 

At  present,  hard  disk  drives  are  considerably  more  expensive 
than  their  floppy  counterparts.  However,  the  performance  figures 
indicate  that  it  would  be  advantageous  to  use  a  hard  disk  instead 
of  a  floppy  disk  if  possible.  To  a  large  extent  this  problem 


Implementing  a  Microcomputer  Database  Management  System 


46 


should  "be  solved  by  new  disks  which  are  .lust  now  entering  the 
marketplace.  Another  possible  solution  would  be  to  share  one 
hard  disk  between  several  processors.  However,  this  configura¬ 
tion  is  likley  to  pose  an  additional  burden  of  a  communications 
network . 

In  summary,  two  immediate  conclusions  can  be  drawn  from  the 
timing  measurements.  First,  it  appears  that  microcomputer  tech¬ 
nology  has  progressed  to  the  point  where  modest  but  realistic 
database  systems  can  be  constructed  on  inexpensive  microcomputer 
hardware.  Second,  it  demonstrates  that  there  exists  an  immediate 
need  for  a  faster  but  inexpensive  secondary  storage  device. 
These  timings  represent  the  performance  obtainable  on  a  system 
today.  It  is  clear  that  future  microcomputer  systems  can  be 
easily  configured  to  achieve  performance  levels  comparable  to 
those  obtained  for  the  larger  configurations. 

9.  Other  Issues 

Two  important  issues  remain  to  be  discussed:  security  and 
fail-recovery.  In  any  computer  system  there  are  two  classes  of 
security:  physical  and  logical.  Physical  security  ensures  that 
the  databases  are  not  accessed  except  through  the  database 
management  system  and  that  any  user  of  the  database  management 
system  is  properly  identified.  Logical  security  ensures  that  the 
database  management  system  allows  a  user  to  perform  only  author¬ 
ized  operations  on  any  item  in  the  database. 

Physical  security  for  a  microcomputer  system  can  be  either 
extremely  easy  or  difficult  to  implement.  If  users  do  not  share 
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databases,  then  each  database  can  occupy  a  separate  floppy  disk 
and  the  security  of  the  floppy  disk  is  the  responsibility  of  the 
user.  If  users  do  share  databases,  then  the  databases  on  floppy 
disks  must  be  available  to  everyone.  Little  can  be  done  to 
prevent  the  floppy  disks  from  being  temporarily  exchanged,  smug¬ 
gled  outside  and  duplicated.  The  compactness  and  portability  of 
the  floppy  disks  contribute  significantly  to  the  problem.  If  the 
microcomputer  console  controls  are  accessible  to  the  user,  then 
the  floppy  disks  can  be  duplicated  on  the  spot  by  bypassing  the 
operating  system.  In  any  event,  without  support  from  hardware 
protection  mechanisms  —  missing  in  the  LSI-11  and  many  other 
microcomputers  —  the  operating  system  can  provide  little  secu¬ 
rity.  The  only  practical  solution  is  to  encode  the  data  so  that 
duplicating  the  floppy  disk  will  not  be  of  much  use. 

Logical  security  is  costly  to  implement.  Each  item  in  the 
database  can  be  tagged  with  security  information  giving  the 
operations  which  any  particular  user  is  authorized  to  perform  on 
that  item.  The  requirements  of  security  clearly  depend  on  the 
application,  for  the  items  of  security  may  range  in  granularity 
from  databases  and  relations  to  individual  tuples  and  attribute 
values.  As  security  information  occupies  much  storage  space  and 
must  be  accessed  along  with  its  associated  item,  a  current  micro¬ 
computer  system  can  probably  support  security  at  the  database  and 
relation  level,  but  not  at  any  finer  level.  Yet,  a  general  secu¬ 
rity  mechanism  should  perhaps  include  at  least  the  tuple  level. 
The  clear  answer  to  this  difficulty  is  superior  hardware. 
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For  fail-recovery,  two  classes  of  failures  can  be  dis¬ 
tinguished:  catastrophic  and  recoverable.  Catastrophic  failures 
destroy  the  file  system  and  the  databases.  Recoverable  failures 
do  not  destroy  anything,  but  may  leave  the  databases  in  an  incon¬ 
sistent  state. 

Catastrophic  failures  happen  when  malfunctioning  hardware  or 
serious  software  errors  cause  an  irrecoverable  loss  of  informa¬ 
tion  on  the  secondary  storage  device.  The  only  safeguards  are 
the  use  of  concurrent  databases  or  database  dumping  and  backup 
coupled  with  transaction  logging.  Concurrent  databases  are 
identical  copies  of  the  master  database  and  they  usually  exist  on 
separate  storage  devices.  Data  updates  are  done  to  all  copies 
simultaneously.  Thus  even  if  one  device  fails,  only  one  copy  of 
the  database  is  lost.  Of  course,  this  does  not  solve  the  problem 
with  faulty  software  which  causes  updates  to  the  wrong  place  in 
all  the  copies.  Database  dumping  and  backup  can  be  done  daily, 
half-daily  or  whenever  there  is  a  need  to  provide  a  recent  copy 
of  the  database.  As  each  user  command  is  entered,  it  is  logged 
out  to  another  device  to  give  a  record  of  the  transaction.  The 
device  may  be  sequential  and  low  speed,  for  example  a  digital 
cassette  tape  recorder.  If  the  database  is  destroyed,  the  most 
recent  backup  copy  is  restored.  This  copy  can  be  brought  up  to 
date  by  re-executing  the  logged  commands. 

Recoverable  failures  happen  when  a  number  of  updates  are 
required  to  transform  the  databases  from  one  consistent  state  to 
another,  but  the  system  fails  before  the  sequence  of  updates  is 
completed.  This  problem  is  particularly  serious  in  systems  where 
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multiple  buffering  schemes  are  used  and  the  failure  of  the  system 
implies  the  loss  of  the  buffer  contents.  One  solution  to  this 
problem  is  to  treat  recoverable  failures  as  catastrophic 
failures.  A  second  solution  is  to  timestamp  each  command  as  it 
is  logged  out  and  also  timestamp  each  item  updated  as  a  result  of 
the  command.  If  the  system  fails,  the  last  logged  command  is 
re-executed,  but  any  item  bearing  the  corresponding  timestamp  is 
not  re-updated.  The  need  to  tag  items  with  timestamps  however, 
may  not  be  appropriate  for  microcomputer  systems  due  to  the  extra 
storage  involved.  A  third  solution  is  to  buffer  images  of  the 
updates  in  an  update  file  and  to  update  the  database  from  this 
file  only  when  the  entire  sequence  of  updates  are  buffered.  If 
the  system  fails  before  the  entire  sequence  is  completed,  the 
update  file  is  simply  discarded  with  no  effect  on  the  database. 
If  the  system  fails  during  updates  from  the  update  file,  the 
database  is  left  inconsistent.  However,  when  the  system  is  res¬ 
tarted,  updates  from  the  update  file  can  resume  from  the  begin¬ 
ning. 

10.  Conclusions 

This  paper  has  outlined  the  inception,  growth  and  develop¬ 
ment  of  a  substantial  piece  of  software  that  is  specifically 
designed  for  a  modest  machine.  The  development  of  such  a  system 
can  proceed  along  a  wide  variety  of  paths.  This  paper  has 
attempted  to  outline  some  of  the  problems  and  insights  gained  in 
the  development  of  a  microcomputer  database  system  and  has  tried 
to  explain  why  certain  choices  were  made  when  a  junction  in  the 
path  was  reached. 
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A  FORM  MANIPULATION  SYSTEM* 


D .  Tsichritzis 

Computer  Systems  Research  Group 
University  of  Toronto 

1  •  Introduction 

Organi za ti ons  find  it  necessary  to  establish  office  pro¬ 
cedures  for  dealing  with  routine  matters.  Office  procedures  are 
often  based  on  paper  forms.  By  paper  forms  we  mean  any  stylized 
piece  of  paper  which  has  some  printed  entries  and  some  fields  for 
filling  in  additional  entries.  There  are  many  examples  of  paper 
forms  and  their  uses,  e.g.,  orders,  bills,  receipts,  inventory 
slips,  etc.  The  office  procedures  associated  with  them  have  some 
common  difficulties.  For  instance,  paper  forms  take  space  to 
store  and  energy  to  move  physically.  In  addition,  paper  forms 
have  a  permanent  structure  which  is  difficult  to  change.  As  a 
result,  office  procedures  based  on  them  are  static  and  they  can 
not  evolve  easily  according  to  changing  needs.  Paper  forms  have 
also  some  advantages.  They  are  not  difficult  to  interpret  and 
fill  in  properly.  People  are  not  allowed  to  make  arbitrary 
changes  on  them.  They  are  often  signed  and  considered  official 
records.  Restrictions  can  be  applied  on  their  duplication.  In 
general,  paper  forms  and  office  procedures  associated  with  them 
are  rather  rigid  but  they  offer  a  high  level  accountability  of 
actions  and  control  of  operations. 


★ 


Invited  paper  NYU  Symposium  on  Automated  Office  Systems,  New  York  University, 
Graduate  School  of  Business,  New  York  City,  May  17-18,  1979. 


53 


A  Form  Manipulation  System 


54 


Computer  systems  have  displaced  or  enhanced  office  manual 
procedures.  They  retain,  for  data  processing,  only  the  raw  data 
associated  with  the  variable  entries  of  the  forms.  They  consoli¬ 
date  or  discard  that  portion  of  paper  forms  that  is  helpful  for 
their  interpretation.  For  instance,  an  inventory  control  system 
displaces  in  this  way  a  manual  system  based  on  order  forms  and 
inventory  slips.  The  paper  forms  are  retained  only  as  a  means 
for  communication  with  the  user.  Input  to  the  system  is  some¬ 
times  effected  through  paper  forms  and  output  is  obtained  in  pa¬ 
per  forms.  Ey  dealing  mainly  with  the  data  on  the  forms  and  not 
the  forms  themselves  computer  systems  lose  some  of  the  advantages 
of  the  manual  form  systems.  Mistakes  are  easier  to  make.  Inter¬ 
mediate  results  are  difficult  to  interpret.  Computer  systems, 
however,  do  not  have  to  abandon  the  form  as  a  concept.  They  can 
deal  with  forms  and  try  to  keep  some  of  the  advantages  associated 
with  them. 


In  this  paper  we  outline  a  system  which  deals  with  forms  and 
the  office  procedures  associated  with  them.  Such  a  system  can  be 
used  as  a  partial  substitute  for  an  office  paper  system.  In  ad¬ 
dition,  it  can  be  used  to  analyze  an  office  paper  system  and  pro¬ 
pose  effective  changes.  Our  system  is  proposed  in  tandem  with  a 
database  management  system.  The  data  on  the  forms  can  be  readily 
accessed  using  data  base  commands.  In  this  way,  we  have  a  dual 
purpose  system.  The  form  manipulation  system  deals  with  the  day 
to  day  operation  of  an  organization.  The  database  management 
system  can  access  the  same  data  as  they  appear  in  the  forms  for 
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further  processing  to  serve  the  needs  of  the  organization. 

2.  Forms 

We  should  distinguish,  first  of  all,  between  forms  and  the 
values  associated  with  them.  A  form  is  not  equivalent  to  the 
values  present  in  its  fields.  Different  operations  are  performed 
on  forms  and  their  field  values.  To  illustrate  the  point,  con¬ 
sider  as  an  example  a  receipt.  A  piece  of  paper  with  the  same 
price  written  on  it  is  not  a  receipt.  For  instance,  the  calcula¬ 
tor  slip  of  a  purchase  is  not  an  official  receipt  valid  to  ex¬ 
change  the  goods.  If  a  receipt  is  not  signed  and  not  marked 
"paid"  it  may  not  he  an  official  receipt.  A  xerox  copy  of  the 
receipt  may  not  he  accepted  as  a  receipt.  A  notarized  copy,  how¬ 
ever,  may  he  accepted  as  an  equivalent  to  the  receipt.  The  point 
is  that  a  paper  form  has  some  properties  which  are  not  retained 
hy  just  capturing  its  fields'  contents. 

If  we  want  a  system  which  is  capable  of  operating  like  a  pa¬ 
per  form  system  we  should  capture  all  the  information  content  of 
the  forms  and  we  should  allow  only  operations  which  can  mirror 
established  paper  form  procedures.  Our  proposed  system,  however, 
will  deal  with  logical  forms.  A  form  in  this  case  is  defined  as 
a  logical  entity  which  has  some  of  the  properties  of  a  paper 
form.  It  corresponds  to  documents  as  defined  in  DDL  [Hammer  et 
al]  or  forms  in  Officetalk  [Newman] . 
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A  form  blank  is  a  character  string  which: 

(a)  has  a  unique  global  name 

(b)  has  a  number  of  fields 

(c)  each  field  has  a  field  name  which  is  unique 
within  the  form  blank 

(d)  each  field  has  a  type  which  specifies  the  kinds  of  values 
that  can  be  filled  in  the  field  including  the  value  empty 

(e)  each  field  has  a  printed  set.  The  printed  set 
is  text  associated  with  the  field 

(f)  each  field  is  designated  as  to  whether  it  can  be  left 
empty  initially  and  whether  it  can  be  updated 

The  fields  and  their  associated  printed  sets  provide  the  ca¬ 
pability  to  structure  the  form  blank  and  associate  some  of  its 
"printed  material"  with  particular  value  entries.  The  form  blank 
may  also  have  printed  sets  not  associated  with  a  particular 
field.  Form  blanks  may  also  be  defined  with  repeating  groups. 
We  will  not  elaborate  on  nice  ways  of  laying  out  forms  through 
graphic  tools  [Newman]  . 

A  form  consists  of  the  following: 

(a)  a  form  blank 

(b)  a  set  of  values  which  are  associated  with  the 
fields  of  the  form  blank 

(c)  a  form  id  identifying  the  form  uniquely 

(d)  a  signature  corresponding  to  the  unique  id  of  the 
creating  station 
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Informally,  a  form  can  "be  represented  as  a  form  "blank  filled 
in  with  a  relation  tuple  of  a  relation  having  as  attributes  the 
field  names  of  the  form,  plus  the  form  id  and  the  creating 
station's  id.  Note  that  the  string  of  characters  representing  a 
"filled"  form  blank  is  not  equivalent  to  the  form. 

Form  operations  are  issued  through  a  station.  A  station  is 
analogous  to  an  office  desk  in  manual  systems.  We  will  not  ela¬ 
borate  on  security  provisions  of  allowing  a  person  to  operate 
from  a  station.  We  will,  however,  assume  that  any  form  operation 
is  issued  from  a  process  associated  with  a  station.  A  station 
has  unique  station  identification  which  we  call  a  signature. 

Forms  are  always  kept  in  special  containers.  A  form  which 
is  not  in  a  container  loses  its  status  as  a  form.  A  form  can 
move  from  container  to  container.  Containers  are  of  two  dif¬ 
ferent  kinds. 

A  form  file  contains  forms  all  of  which  have  the  same  form 
type.  A  form  file  corresponds  to  a  form  blank  together  with  a 
relation  for  the  values  of  the  forms  and  the  associated  form  id's 
and  creating  station  signatures.  A  form  file  can  contain  forms 
with  different  creating  station  signatures.  A  form  file  has  a 
form  (file)  key  which  is  a  set  of  field  names  of  the  form  blank. 
A  form  key  is  minimal  and  it  uniquely  identifies  a  form  in  the 


form  file. 
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A  form  heap  is  a  container  which  can  contain  forms  without 
any  restrictions.  A  key  is  not  defined  for  form  heaps.  Heaps 
are  extremely  useful  as  mailboxes  (in-trays)  for  general  messages 
sent  using  forms.  A  dossier  is  a  temporary  heap.  It  can  assem¬ 
ble  a  set  of  forms.  Forms  do  not  need  to  be  moved  physically  to 
the  dossier.  The  dossier  only  serves  as  a  logical  collection 
mechanism  for  forms.  Forms  still  belong  to  different  files  or 
heaps.  Dossiers  can  be  used  to  create  a  view  of  a  set  of  files 
and  heaps  by  retaining  a  subset  of  the  forms  and  ordering  them  in 
a  desired  manner. 

Each  form  file  or  form  heap  has  a  unique  owner  station.  In 
addition,  each  form  file  or  heap  has  an  access  control  list  of 
other  stations  which  can  issue  operations  on  it. 

3.  Form  Operations 

Form  operations  should  allow  users  to  enter,  read,  move  and 
generally  manipulate  forms.  At  the  same  time  operations  are  res¬ 
tricted  in  order  to  retain  some  of  the  desired  properties  of  pa¬ 
per  form  systems.  The  operations  correspond  to  special  function 
keyboard  buttons  for  the  unsophisticated  users.  The  operations 
proposed  are  explained  only  in  gross  terms.  Their  detailed  syn¬ 
tax  is  implementation  dependent.  Operations  are  grouped  accord¬ 
ing  to  implementation  considerations.  In  a  users  manual,  howev¬ 
er,  they  should  be  grouped  according  to  office  procedures,  i.e., 
form  filling,  finding,  filing,  mailing,  printing  and  processing. 
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*  (la)  Find  form  in  station  using  form  id 

All  compatible  files  and  heaps  of  a  station  are  searched.  The 
command  returns  the  name  of  the  form  file  or  heap  where  the  form 
is  and  makes  the  form  current. 

*  (lb)  Find  form  in  form  file  using  key  value 

A  form  is  made  current  given  a  form  file  name  and  a  value  of  the 
key  of  that  form  file.  This  command  can  be  issued  only  for  form 
files  (heaps  have  no  key). 

*  (lc)  Find  {first  I  last 'next 'previous}  in  form  file  or  heap 

where  optional  selection  clause 

The  firs t ' las t J nex t ' previous  form  is  made  current  on  a  given  form 
file  or  heap.  A  boolean  selection  of  field  conditions  can  be 
specified. 

*  (2a)  Attach  form  to  dossier 

A  found  form  is  logically  attached  to  a  dossier  using  this  command 

*  (2b)  Sort  form  file  according  to  field  name 

This  command  is  only  used  for  rearranging  form  files.  Sorting 
heaps  presents  a  problem  since  forms  may  not  have  a  common  attri¬ 
bute  . 

*  (2c)  Sort  heap  according  to  form  groups 

In  heaps,  forms  of  the  same  form  type  are  sorted  within  their 
group . 

*  (3a)  Read  field  values  of  form 

This  is  the  mechanism  by  which  the  form,  system  interfaces  with 
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other  programs  for  further  processing.  All  read  commands  operate 
on  the  current  form. 

*  (3d)  Display  form 

Forms  are  displayed  on  CRT's,  or  images  of  forms  are  printed. 

*  (3c)  Display  form  file,  dossier,  or  heap 

Display  of  form  files,  dossiers  and  heaps  on  CRT  is  accomplished 
through  scrolling.  Printing  implies  hulk  printing  of  all  con¬ 
tents  of  file,  heap,  or  dossier. 

*  (4a)  Move  current  form  of  file/heap  as  {f i rs t ! las t j next ! previ ous } 

of  other  file/heap. 

This  command  presupposes  the  existence  of  a  network  facility 
which  can  send  forms  as  messages  to  other  stations.  Move  com¬ 
mands  have  a  special  status  and  different  security  checks  when 
either  the  origin  or  the  destination  of  the  form  is  different 
than  the  station  issuing  the  command.  A  distinction  can  he  made 
between  local  and  global  moves  using  the  terms  MOVE  and  MAIL. 

*  (4b)  Move  dossier  or  file  or  heap  to  another  file  or  heap 

This  is  a  bulk  move  of  many  forms  together  and  is  implemented  by 
shipping  files.  The  command  is  restricted  to  three  cases:  "file 
to  file",  "homogeneous  dossier  to  same  type  file"  and  "file,  dos¬ 
sier  or  heap  to  heap". 

*  (5a)  Locate  form  given  its  creating  station  and  unique 

identification  number 

This  command  returns  the  current  station  in  which  a  form  resides. 
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The  unique  identification  number  is  presumably  known  to  the  in¬ 
terested  user. 

*  (5b)  Trace  form  given  its  creating  station  and  unique 

identification  number 

This  command  returns  a  list  of  all  stations  that  a  form  has 
visited  including  the  current  station.  Trace  commands  are  impor¬ 
tant  for  auditing. 

*  (6a)  Assign  field  values  to  form  blank 

Data  entry  is  accomplished  by  displaying  a  form  and  assigning 
values  to  its  fields. 

*  (6b)  Enter  form  as  {first  I  last | next iprevious}  of  current  in 

form  file  or  heap 

A  displayed  form  is  stamped  with  the  signature  of  the  creating 
station  and  is  entered  in  the  system.  It  also  gets  a  unique 
identifying  number.  Once  a  form  is  entered  it  can  not  be  des¬ 
troyed,  not  even  by  its  creating  station.  Our  system  does  not 
have  a  destroy  command.  A  form  is  taken  out  of  circulation  by 
nailing  it  to  a  disposal  station. 

*  (6c)  Assign  field  value  to  form  field 

Some  form  fields  can  be  left  empty  at  create  time.  These  fields 
can  be  assigned  values  later  on.  The  signature  of  the  station 
entering  the  value  is  retained. 

*  (6d)  Modify  field  value 

This  command  is  only  successful  when  the  field  has  been  defined 
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as  updatable  in  the  form  blank  definition.  The  signature  of  the 
last  station  changing  the  value  is  retained. 

Rearranging  information  on  forms  can  be  accomplished  by  pro¬ 
gramming  application  oriented  user  commands.  These  "user- 
buttons”  are  implemented  by  small  programs  dispersed  with  form 
commands.  The  programs  use  information  contained  in  the  fields 
of  existing  forms.  Note  that  the  forms  used  are  not  consumed. 
The  new  forms  are  generated  with  different  station  signatures. 

*  Expand  form  by  adding  new  fields 

*  Truncate  form  by  scratching  out  some  fields 

*  Concatenate  two  forms  by  merging  their  fields 

*  Copy  form 

This  command  creates  an  exact  copy  of  a  form  except  it  has  a  dif¬ 
ferent  signature.  If  the  station's  signature  is  recognized  as  a 
valid  one  the  new  copy  (notarized  copy)  can  have  the  same  effect 
as  the  original.  We  do  not  have  a  special  duplicate  command 
which  generates  an  exact  copy  including  signature.  In  this  way 
we  retain  the  differentiation  between  an  original  and  a  copy 
which  exists  in  manual  systems. 

*  Destroy  form 

A  destroy  form  is  implemented  as  a  move  to  a  special  disposal 
station.  The  station  does  not  really  destroy  the  form  but  merely 
takes  it  out  of  circulation.  The  form  stops  clogging  up  the 
files  of  the  other  stations,  but  it  may  still  be  needed  for  back- 
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up  or  auditing  purposes.  The  disposal  station  can  act  as  a 
"shredder"  or  an  "auditing”  station. 

Operations  can  also  he  defined  analogous  to  relational 
operations  which  work  on  a  set  of  forms  in  a  file.  For  instance, 
joins,  projections,  selections  can  he  defined  on  form  files. 
However,  we  consider  these  operations  difficult  to  understand  and 
not  necessary  for  office  personnel.  These  operations  are  readily 
available,  however,  hy  viewing  the  data  on  the  forms  as  data  base 
objects  and  issuing  data  base  commands  on  them. 

All  commands  are  issued  from  stations.  Stations  are  res¬ 
tricted,  at  station  creation  time,  as  to  the  types  of  operations 
that  they  can  perform.  By  restricting  stations  to  only  certain 
specific  operations  and  recognizing  their  signatures  as  valid  for 
only  certain  specific  actions  we  can  define  specialized  stations. 
For  instance: 

(1)  Query  station  A  station  which  can  find  and  read  forms. 

(2)  Mail  station  A  station  which  can  find,  rearrange  and  ship 

forms . 

(3)  Entry  station  A  station  which  can  assign  field  values 

and  enter  forms  in  its  files/heaps. 

(4)  Library  station  A  station  which  can  find  and  ship  forms. 

Other  stations  ship  forms  to  it  for  safe 
keeping . 

(5)  Control  station  A  station  which  can  find,  locate  and 

trace  forms  on  everybody's  files/heaps. 
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(6)  Secretary  station  A  station  whose  signature  is  recog¬ 

nized  to  create  good  dossiers  and  cut 
and  paste  material  in  new  forms. 

(7)  Copy  station  A  station  whose  signature  is  recognized 

as  creating  exact  copies. 

(£)  Disposal  station  A  station  to  whose  files/heaps  every¬ 
body  can  ship  forms,  lut  from  which  nobody 
can  take  forms. 

4.  Pilot  Implementation 

A  pilot  form  manipulation  system  is  being  implemented  at  the 
University  of  Toronto  using  LSI-ll's  and  a  central  PDP-11/45  fa¬ 
cility.  The  LSI-ll's  have  64k  byte  memory  and  floppy  disks.  The 
PDP-11/45  has  many  peripherals  including  fancy  graphic  displays 
and  a  typesetting  support  facility.  The  LSI-ll's  operate  under  a 
modified  version  of  MINI  UNIX  [Lycklama] .  The  PDP-11/45  operates 
under  regular  UNIX  [Thompson  and  Ritchie].  Stations  are  imple¬ 
mented  on  the  LSI  ll's.  Some  special  stations  are  also  imple¬ 
mented  as  processes  running  on  the  PDP  11/45  through  terminals. 
The  PDP  11/45  serves  as  a  central  archival  facility,  as  a  print¬ 
ing  facility  and  as  a  mail  repository.  It  also  provides  control 
of  the  overall  system. 

The  form  manipulation  system  is  implemented  alongside  MRS,  a 
relational  data  base  system  which  has  already  been  implemented  on 
LSI-ll's  and  PDP-11 's  [Hudyma  et  al] .  The  two  systems  use  the 
same  conventions  for  files  and  indexes.  In  this  way  data  can 
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pass  from  the  form  manipulation  system  to  the  data  "base  system 
and  vice  versa  easily. 

Each  form  type  corresponds  to  a  UNIX  character  file  resident 
in  the  PDP-11.  When  a  station  running  on  a  LSI -11  needs  a  form 
blank  it  requests  a  form  type  from  the  central  site  and  keeps  it 
local  in  a  table.  Form  blanks  are  designed  using  extensive 
graphical  tools  available  on  the  central  PDP-11/45  facility.  Us¬ 
ing  this  facility  the  layout  formats  for  the  display  and  the 
print ing  are  set . 

Each  form  gets  a  form  id  when  it  is  entered  in  the  system. 
The  form  id  is  implemented  as  a  (station,  blank,  number)  triplet. 
The  number  is  provided  by  a  counter  associated  with  the  form 
blank  in  the  entering  station. 

Form  files  are  implemented  as  relations  with  one  attribute 
per  form  field,  plus  an  attribute  for  the  form  id  which  includes 
the  creating  station  signature.  Form  fields  which  can  be  updated 
or  entered  later  get  an  extra  attribute  for  encoding  of  the  sta¬ 
tion  signature  issuing  the  operation.  Form  heaps  are  implemented 
as  a  number  of  form  files.  The  forms  in  a  heap  are  grouped  by 
their  type.  If  a  form  of  a  new  type  enters  a  heap  it  implies  the 
creation  of  a  new  form  file  associated  with  the  heap.  Indexes  to 
form  files  and  heaps  are  implemented  using  E-trees  as  in  the  MRS 
system.  Dossiers  are  implemented  as  pointer  arrays  pointing  to 
records  in  different  files  or  heaps. 
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There  are  two  types  of  form  files/heaps,  private  and  shared. 


All  LSI-11 

based  form 

files 

and 

heaps  are 

private. 

Private 

files/heaps 

allow  operations 

only 

by  their 

owner . 

Sha  red 

files/heaps 

are  always 

PDP-11/45 

resident. 

Shared 

files  and 

heaps  allow  find,  rearrange  and  fetch  form  commands  only  from 
their  owner.  Shared  files  and  heaps  allow  ship  command  s  from  a 
number  of  stations  specifically  listed  in  an  access  control  list. 
No  other  commands  are  allowed  on  shared  files/heaps.  Shared 
files/heaps  are  essentially  in-trays  for  stations.  A  station  can 
search  and  pick  out  forms  from  its  own  in-tray(s).  It  can  also 
put  forms  at  the  bottom  of  other  stations  in-trays  provided  it  is 
authorized  to  put  forms  there.  A  form  sent  will  only  be  received 
by  a  station  when  the  receiver  station  picks  up  its  mail.  A  re¬ 
quest  for  a  form  is  essentially  another  form  which  is  sent  to  the 
appropriate  station.  Forms  are  always  put  at  the  bottom  of  the 
in-tray.  Since  all  in-trays  are  associated  with  the  central  site 
we  do  not  have  the  problem  of  trying  to  send  mail  to  a  discon¬ 
nected  or  inoperative  station  on  a  LSI-11.  In-trays  do  not  easi¬ 
ly  overflow  since  they  are  PDP-11/45  based. 

Move  operations  within  a  station  including  fetching  forms 
from  an  in-tray  are  not  controlled.  A  log  of  operations  is  not 
kept.  A  station  is  a  uni-user  environment  and  operates  freely  in 
its  local  environment.  Move  operations  are  controlled  and  a  log 
is  kept  only  when  a  station  ships  a  form  to  another  station's 
in-tray.  Using  these  logs  the  central  site  implements  locate  and 
trace  commands.  For  each  move  operation  the  log  records  the  form 
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id  plus  the  sending  station  and  receiving  station. 

A  user  moves  a  form  to  a  private  file/heap  before  he  can  is¬ 
sue  any  read,  display  and  other  manipulation  commands.  In  this 
way  interference  is  avoided.  Once  a  form  is  in  a  private 
file/heap  it  is  not  protected  and  there  is  no  concurrency  provi¬ 
sion.  In  our  environment  only  one  station  has  access  to  it.  A 
current  form,  or  a  form  pointed  to  by  a  dossier  is  not  frozen. 
It  can  still  be  moved  updated  or  manipulated. 

Forms  are  found  using  B-tree  indexes,  when  necessary,  for 
searching  purposes.  A  currency  indicator  is  set  if  a  form  is 
found.  Each  form  file  has  only  one  currency  indicator.  In  our 
environment  only  one  station  is  permitted  to  find  a  form  in  any 
private  or  even  shared  file/heap.  Shared  files/heaps  allow  other 
stations  to  put  forms  in  them,  but  not  to  search  them. 

A  form  is  filled  by  displaying  a  form  blank  on  a  screen. 
The  appropriate  field  values  are  given  and  a  form  id  is  displayed 
by  the  system.  The  station  user  can  keep  the  form  id  for  future 
reference.  After  a  form  is  filled  it  can  be  entered  with  a 
separate  command  into  a  form  file/heap. 

Display  of  found  forms  is  effected  by  combining  the  charac¬ 
ter  strings  of  the  form  blank  with  the  field  values  obtained  from 
the  form  file.  When  a  form  is  displayed  there  is  no  interfer¬ 
ence,  since  no  other  station  can  operate  on  this  form.  The  form 
resides  in  a  private  file/heap.  Display  is  effected  using  the 
special  format  defined  during  the  form  blank  design.  The  form's 
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id  is  also  displayed. 

A  form  read  operation  is  effected  by  transferring  the  field 
values  to  a  MRS  data  base  relation  for  further  processing. 
Printing  of  forms  is  effected  by  moving  them  to  an  in-tray  of  a 
special  station  operating  on  the  central  site.  Printing  format 
is  already  defined  during  form  blank  design.  The  printing  sta¬ 
tion  prints  either  on  line  or  periodically  what  exists  in  its 
in-trays.  A  copy  station  is  implemented  as  a  special  station  on 
the  PDP-11/45.  A  destroy  station  is  also  implemented  as  a  spe¬ 
cial  station  on  the  PDP-11/45  with  an  effectively  infinite  in- 
tray  for  removing  forms  from  circulation. 

Our  pilot  implementation  is  carried  out  by  Christine  Cheung 
and  Simon  Gibbs  using  the  MRS  system  already  developed  and  avail¬ 
able  graphical  form  filling  and  displaying  tools.  We  have  taken 
a  simplified  approach  to  ease  the  implementation  problems.  We 
plan  to  move  eventually  to  an  environment  in  which  the  LSI-11 
oriented  stations  can  exchange  forms  directly  through  a  local  or 
global  network  without  a  central  site.  For  the  time  being,  how¬ 
ever,  the  central  site  offers  many  advantages.  It  helps  in  the 
coordination  of  the  activity.  In  addition,  it  does  printing,  ar¬ 
chiving  and  other  services  which  can  not  be  easily  distributed. 

5.  Concluding  remarks 

We  propose  some  ideas  for  form  manipulation  and  we  are 
currently  implementing  a  pilot  project.  The  pilot  system  should 
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"be  ready  "by  the  end  of  summer  1979. 

The  form  manipulation  system  is  implemented  alongside  a  da¬ 
tabase  management  system.  The  user  can  use  form  manipulation 
commands  to  work  on  forms  as  documents.  At  the  same  time,  he  can 
use  data  base  commands  to  work  on  the  data  of  the  forms  and  pro¬ 
cess  them  to  support  the  decision  making  of  an  organization.  The 
combined  system  both  captures  the  day  to  day  activity  and  sup¬ 
ports  the  long  range  needs  of  an  organization. 

Our  form  manipulation  system  mirrors  the  existing  procedures 
and  conventions  used  for  paper  systems.  In  this  way,  it  will  be 
less  foreign  and  threatening  to  prospective  users.  User  accep¬ 
tance  is  the  key  issue.  Hardware  utilization  and  fancy  architec¬ 
tures  are  not  critical.  Eventually  we  may  be  able  to  carefully 
urge  the  users  towards  more  elegant  technical  solutions,  or  even 
more  powerful  tools.  Eut  for  the  time  being  we  are  implementing 
what  the  users  are  accustomed  to. 

Form  manipulation  systems  can  be  used  not  only  as  a  partial 
substitute  for  paper  systems,  but  also  to  analyze  them.  They 
can,  in  essence,  simulate,  in  part  or  in  whole,  existing  manual 
systems,  or  analyze  proposed  changes.  We  are  working  on  form 
flow  analysis  with  techniques  which  apply  equally  well  to  compu¬ 
terized,  manual  or  hybrid  systems  [Tsichritzis] . 

It  is  very  difficult  to  speculate  on  how  users  would  react 
to  form  manipulation  systems.  We  are  not  dealing  with  sophisti¬ 
cated  users  with  a  good  technical  background.  The  prospective 
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users  may  be  reluctant  to  learn  to  use  the  systems.  They  will 
feel  constrained  to  sorre  very  strict  office  procedures  as  en¬ 
forced  by  the  tools  they  use.  In  addition,  the  users  may  per¬ 
ceive  a  threat  to  their  .lob  security  and  it  is  difficult  to  blame 
them.  Once  their  freedom  of  operation  is  constrained,  their  role 
can  be  well  defined.  Once  their  role  is  well  defined  they  can 
potentially  be  replaced  by  a  machine.  However,  forms  and  form 
systems  always  require  interpretation  and  exception  handling. 
Humans  might  always  be  better  and  more  efficient  at  this  than 
machine  s . 

Form  manipulation  systems  as  part  of  office  automation  are 
bound  to  be  a  mixed  blessing  for  clerks.  They  are,  in  essence, 
given  very  elegant  tools  which  will  increase  their  productivity 
but  may  constrain  their  freedom  of  operation.  The  clerks  in  or¬ 
ganizations  may  react  very  negatively.  For  the  people  at  large, 
it  may  be  a  different  story.  Fureaucracies  are  getting  threaten¬ 
ing  enough  that  anything  which  threatens  them  and  controls  them 
may  be  perceived  as  a  blessing  for  the  common  man. 
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1.  Introduction  and  definitions 

Many  people  currently  work  in  service  industries.  Their  main  occupa¬ 
tion  is  to  process  information.  Organizations  find  it  necessary  to  establish 
ground  rules  and  procedures  for  gathering,  processing  and  acting  on  informa¬ 
tion.  These  procedures  are  often  not  rigorously  defined.  The  humans  following 
them  can  provide  the  necessary  added  insight  and  flexibility  to  remedy  any 
shortcomings  in  the  procedures.  However,  if  some  of  the  procedures  are  car¬ 
ried  out  by  machines,  the  actions  and  the  flow  of  information  need  to  be  exact. 
Errors  and  system  peculiarities  may  not  be  detected  and  eliminated  by  alert  hu¬ 
mans.  They  should  therefore  be  avoided  by  careful  design  and  analysis  of  office 
procedures  and  information  flow. 

Many  office  procedures  are  based  on  forms.  For  instance,  we  file  an  in¬ 
come  tax  form,  an  expense  account  form,  a  passport  application  form^etc.  The 
office  procedures  specify  what  path  the  forms  follow  and  what  actions  Eire  taken 
as  a  result  of  a  form’s  presence.  A  form  has  an  origin,  a  series  of  stations  where 
it  is  manipulated  and  a  destination.  Suppose  that  some  of  the  stations  can  be 
computerized  and  at  some  point  a  paper  form  becomes  an  image  of  a  paper 
form  in  a  computer.  In  this  event,  a  form's  path  should  be  carefully  analyzed  to 
detect  and  remove  potential  processing  problems.  For  instance,  we  do  not  ex- 
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pect  a  manual  form  treated  by  humans  to  get  into  an  infinite  loop  in  an  office. 
(Although  sometimes  it  seems  that  exactly  this  happens.)  Somebody  •will  eventu¬ 
ally  detect  such  a  loop  in  a  manual  system.  Computers  do  not  have  the  same  in¬ 
sight.  If  we  are  not  careful,  a  form  may  circulate  in  the  system  forever.  To 
avoid  such  problems,  we  propose  a  model  for  the  analysis  of  form  manipulation 
systems.  The  model  will  also  provide  insight  into  the  proper  design  of  office  pro¬ 
cedures  for  working  on  forms. 

Consider  a  system,  manual,  computerized  or  hybrid  which  manipulates 
forms.  Forms  can  be  thought  of  as  paper  forms  or  as  logical  entities  manipulat¬ 
ed  by  computers  which  have  the  same  properties  and  operations  as  paper 
forms.  Forms  are  entered,  moved  around,  looked  at.  stamped  and  signed  and 
eventually  reach  a  final  destination.  Forms  are  manipulated  through  stations.  A 
station  can  be  a  clerk’s  desk  in  a  manual  system,  a  process  in  a  large  computer 
system  or  a  desk  computer.  A  station  can  issue  operations  on  forms  and  it  can 
keep  files  of  forms.  A  form  cannot  be  destroyed,  but  we  can  take  it  out  of  circu¬ 
lation  by  shipping  it  to  an  archival  or  disposal  station.  We  will  also  assume  that  a 
form  cannot  be  duplicated.  Its  information  can  be  copied  but  the  copy  will  be 
another  form  distinguishable  from  the  original.  Each  form  and  station  are 
uniquely  identified.  We  will  not  elaborate  in  this  paper  on  operations  on  forms, 
or  on  computer  systems  that  manipulate  forms  [Hammer  et  al.Tsichritzis].  We 
will  look  at  the  flow  of  forms  among  stations.  Our  goal  is  to  analyze  form  flow,  to 
indicate  the  loads  of  work  on  stations  for  form  manipulation  and  to  investigate 
station  and  flow  reorganization.  Our  observations  are  applicable  not  only  to 
computerized  form  manipulation  systems  but  also  to  manual  systems. 

Consider  a  set  of  N  stations  manipulating  forms.  We  will  assume  for 
simplification  purposes  that  the  stations  are  labelled  by  integers  1,2,3,...,N.  A 
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station  diagram  is  defined  as  a  directed  graph  with  stations  as  nodes  (fig.  la). 
There  is  a  directed  adge  from  station  i  to  station  j,  if  station  i  sends  any  forms 
to  station  j.  Each  directed  edge  is  labelled  with  the  types  of  forms  6 
that  are  being  sent.  Notice  that  there  is  a  duality  between  sending  forms  and 
sending  requests  to  operate  on  forms.  To  avoid  any  confusion  we  will  assume 
that  all  operations  apply  only  to  forms  local  to  a  station.  That  is,  a  form  moves 
to  a  station  before  it  is  operated  on  from  that  station.  We  will  augment  the  sta¬ 
tion  diagram  with  two  special  stations.  A  source  station  a  and  a  sink  station  cj. 

Any  form  originating  in  a  station  i  will  be  thought  of  as  being  sent  to  i  from  the 

V 

source  station  a.  Any  form  which  is  deposited  to  a  final  repository  station  t  will 
be  thought  of  as  being  sent  from  t  to  the  sink  station  cj. 

A  form  path  or  simply  a  path,  is  a  sequence  of  stations  corresponding 
to  a  directed  path  in  the  station  diagram.  The  path  ( i,j,k,...,r,s,t )  is  interpreted 
as  coming  from  station  i  to  station  j  passing  stations  j,k,...,r,s.  and  going  to  sta¬ 
tion  t.  The  special  stations  a  and  cj  can  only  appear  at  the  beginning  and  the 
end  of  a  path  respectively.  A  form  path  can  be  infinite.  A  form  path  is  cyclic  if  it 
is  of  the  form  (i,j,...,(k,...,l)*)  where  the  sequence  of  stations  k,...,l  is  repeated 
indefinitely. 

Consider  two  form  paths  with  a  common  station.  The  two  paths  will  be 
dependent  with  respect  to  that  station  if  there  are  at  least  two  forms  passing 
from  the  station  such  that  the  progress  of  one  depends  on  the  other.  Notice 
that  there  is  no  uncertainty  about  the  destination  of  each  form.  Both  paths  are 
linear.  Dependency  means  that  a  form  waits  for  another  before  it  proceeds.  A 
station  which  involves  some  dependency  will  be  called  a  coordination  station. 
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A  form  trace,  or  simply  a  trace  is  a  sequence  of  stations  which  a  partic¬ 
ular  form  historically  visits.  A  trace  is  a  path  that  a  form  actually  takes.  There 
may  be  some  paths  that  forms  never  take  and,  therefore,  do  not  correspond  to 
traces.  A  trace  always  starts  with  the  special  source  station  a.  It  ends  with  the 
special  sink  station  cj  only  if  it  is  finite.  Infinite  and  cyclic  traces  correspond  to 
infinite  and  cyclic  paths  that  forms  actually  take.  A  trace  is  repeating  if  at  least 
one  station  repeats  itself.  A  trace  may  be  repeating  without  being  cyclic  or 
infinite.  We  can  argue  that  it  is  bad  bureaucratic  policy  to  have  forms  circulat¬ 
ing  forever.  We  will  try,  therefore,  to  avoid  infinite  traces.  There  are  situations, 
however,  in  manual  systems  where  a  form  comes  back  to  the  same  station.  For 
instance,  a  form  is  sent  to  a  clerk  but  it  requires  some  authorization  before  he 
can  act  on  it.  Repeating  traces  will,  therefore,  be  permitted. 

A  form  class  is  a  set  of  forms  which  have  the  same  form  type  and  the 
same  trace.  Corresponding  to  the  form  class  is  a  class  path  which  is  a  sequence 
of  stations  depicting  the  common  trace.  It  is  desirable  that  all  forms  in  the  sys¬ 
tem  belong  to  one  of  a  finite  number  of  form  classes.  This  property  implies  well 
designed  office  procedures.  Each  form’s  progress  is  not  arbitrary  but  follows  a 
preset  pattern.  A  second  useful  property  is  that  the  pattern  be  predictable.  A 
form  has  a  predictable  trace  if  there  exists  an  algorithm  to  obtain  the  trace  on 
the  basis  of  the  form’s  origin  and  contents.  The  same  property  can  be  formulat¬ 
ed  by  saying  that  each  form  class  is  a  recursive  set. 

Consider  a  finite  set  of  form  classes  and  their  form  paths  as  they  exist 
in  a  system,  and  the  combination  of  all  these  form  paths  in  one  diagram.  Each 
node  in  the  diagram  corresponds  to  a  (station,  class)  pair.  All  (station,  class) 
pairs  of  the  same  station  are  clustered.  All  (station,  class)  pairs  of  the  same 
class  are  connected  with  directed  edges  of  the  form  path  of  this  class.  To  depict  > 
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repeating  stations  we  distinguish  the  next  occurrence  of  the  same  station  in  the 
same  path  with  a  different  node.  If  a  path  is  cyclic  we  outline  the  cycle.  We  will 
call  such  a  diagram  a.  form  flow  diagram  (fig.  2a).  It  depicts  the  flow  paths  that 
forms  take  in  each  class. 

The  station  diagram  depicts  only  stations  and  what  forms  are  sent  and 
received  by  them.  The  form  flow  diagram  describes  in  detail  the  form  flow  and 
characterizes  each  class.  Many  different  form  flow  diagrams  can  correspond  to 
the  same  station  diagram.  In  the  following  section  a  procedure  will  be  outlined 
to  obtain  the  pertinent  form  flow  diagram. 

2.  Form  flow  diagrams 

Consider  a  form  manipulation  system.  It  would  be  nice  to  obtain  the 
form  flow  diagram  associated  with  the  system.  We  assume  some  information 
which  we  can  get  by  local  interviews  in  each  station.  For  instance,  we  assume 
knowledge  about  the  forms  a  station  receives,  the  forms  it  sends  out  and  the 
forms  which  are  dependent  with  respect  to  that  station.  A  method  is  outlined 
for  obtaining  the  form  classes  and  associated  paths.  A  way  is  suggested  to  esti¬ 
mate  the  volume  of  forms  in  each  class  on  the  basis  of  the  distribution  of  form 
field  values.  Our  analysis  is  based  on  certain  assumptions. 

1)  Consistency.  If  a  form  reaches  a  station  with  the  same  values  in  its  fields  and 
the  same  "history"  as  a  previous  form,  then  it  will  receive  the  same  treatment  as 
the  previous  form  including  what  station  it  goes  to  next.  That  is,  a  station  is 
consistent  in  its  treatment  of  forms.  Its  action  on  the  form  follows  predefined 
rules.  A  user  operating  from  a  station  can  take  other  personal  and  arbitrary  ac¬ 
tions,  but  we  assume  that  they  do  not  affect  the  form’s  progress  in  the  system. 
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2)  Immediate  history.  The  progress  of  a  form  does  not  depend  on  its  complete 
"history"  but  on  its  immediate  "history”.  That  is,  the  complete  path  of  a  form 
up  to  a  station  is  unknown  to  that  station.  The  only  information  that  a  station 
has  about  a  form  is  contained  in  its  field  values  and  knowledge  about  where  the 
form  comes  from.  We  exclude  the  possibility  that  a  complete  path  is  encoded  in 
the  form's  field  values.  This  restriction  can  be  effected  by  excluding  the  possi¬ 
bility  of  updating  field  values  on  which  the  form’s  trace  depends. 

3)  Determinacy.  There  is  no  indeterminacy  with  respect  to  a  form’s  ultimate 
destination.  Either  a  form  will  reach  a  final  station,  or  it  will  have  an  infinite 
trace.  That  is,  if  a  form  is  dependent  on  another  form,  the  other  form  will  even¬ 
tually  arrive.  In  addition,  when  a  form  reaches  a  station  it  will  either  stay  there 
forever,  or  it  will  go  next  to  one  particular  station. 

Consider  a  station  diagram  S.  For  each  form  type  q  there  is  a  portion 
of  the  station  diagram  Sq  which  applies  to  that  form  type.  For  the  time  being  we 
will  not  consider  the  coordination  problems.  We  can,  therefore,  work  on  Sq 
separately  for  each  form  type  q.  In  the  sequel  we  will  drop  the  subscript  q  from 
any  notation  unless  it  is  absolutely  necessary. 

We  define  for  each  station  j  a  set  of  characteristic  functions  F(i,j,k) 
(fig.  lb).  The  characteristic  function  F{i,j,k )  is  true  or  false  and  determines  the 
forms  which  when  received  from  station  i  at  station  j  go  to  station  k.  The  func¬ 
tion  F(a,i,j)  characterizes  the  forms  originating  in  station  i  and  going  to  station 
j.  The  function  F(ij,  cj)  characterizes  the  forms  which,  coming  from  station  i  to 
station  j,  stay  there  forever.  Our  consistency  and  immediate  history  assump¬ 
tions  imply  that  the  characteristic  functions  are  predicates  with  respect  to  the 
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The  characteristic  functions  for  each  station  cannot  be  completely  ar¬ 
bitrary.  Our  determinacy  assumption  implies  that  there  are  no  two  F(i,j,k), 
F{i,j,l)  which  are  satisfied  by  the  same  form.  In  addition,  we  expect  some  flow 
continuity  rules.  That  is,  if  a  form  arrives  at  a  station  it  will  either  go  some¬ 
where,  or  it  will  stay  there.  We  can  take  two  approaches  for  defining  charac¬ 
teristic  functions  that  preserve  continuity.  In  the  first  approach,  we  assume 
that  a  station  is  prepared  to  receive  any  form  of  a  given  type  from  another  sta¬ 
tion  provided  it  is  consistent  with  the  station  diagram.  That  is,  a  station  cannot 
arbitrarily  refuse  to  receive  a  form.  In  this  case,  the  characteristic  functions 
defined  satisfy  certain  rules. 

For  every  station  j,  and  for  every  station  i  sending  forms  to  j  according 
to  the  station  diagram  (including  a)  and  for  all  stations  k  receiving  forms  from  j 
according  to  the  station  diagram  (including  cj  ): 

SfM)=  1 

k 

That  is,  all  the  forms  sent  from  i  to  j  will  either  stay  in  j  or  will  go  to  some  other 
station  k.  We  use  the  notation:  0  for  false,  1  for  true,  +  for  disjunction  and  .  for 
conjunction. 

In  a  second  approach  a  station  may  send  a  form  to  another  station 
which  refuses  to  accept  it.  In  this  way,  the  sending  station  ends  up  with  the 
forms  which  it  keeps  and  the  forms  that  the  receiving  stations  send  back.  This 
situation  implies  that  the  functions  F(i,j,  cj)  are  redefined  to  include  the 
rejected  forms.  Alternatively  the  function  F(i,j,cj)  can  be  left  completely 
undefined  and  they  are  calculated  as  the  difference  between  the  forms  a  station 
receives  minus  the  forms  that  it  sends  away. 

cj)  =  1-  £  F  (i.j.k) 

k,  except  a 

That  is  the  functions  F(i,j,  cj)  are  defined  to  satisfy  the  continuity  rule  as 
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outlined  in  the  previous  approach. 

The  characteristic  functions  give  us  the  local  behaviour  of  each  station. 
They  can  be  presumably  obtained  by  interviews  outlining  the  function  and  pro¬ 
perties  of  each  station.  We  are  interested,  however,  in  obtaining  information  on 
form  flow  in  the  whole  system.  We  will  outline  a  procedure  to  define  the  form 
classes  and  their  associated  paths  in  terms  of  the  characteristic  functions.  We 
present  a  series  of  mini-theorems  which  we  will  call  statements,  since  they  are 
not  very  hard  to  prove.  Consider  a  path  i,j,k,...,Tn,n,t  interpreted  as  coming 
from  station  i  passing  stations  j,k,...,m,n  and  going  to  station  f.  An  extension  of 
the  characteristic  functions  F  is  defined  in  a  natural  way  for  all  stations  i  includ¬ 
ing  the  source  station  a,  for  all  stations  j,k,...,7n,n  and  for  all  stations  t  including 
the  sink  station  cj. 

F(i,jJk,...,m,n,t)  =  F(i,j,k . m,n)  F(jn,n,t) 

Statement  1  If  a  form  satisfies  F(i,j,k,...,7n,n,t),  then  if  the  form  is  sent  from 
station  i  to  station  j,  it  will  follow  the  path  j,k,...,mlTi  and  then  go  to  station  t. 

Statement  2  If  a  form  sent  from  station  i  to  j  follows  the  path  j,k,...,7Tit7i  and 
then  goes  to  station  t,  then  it  satisfies  F  (i,j,k,...,7n,n,t ). 

It  follows  that  the  extension  function  F(i,j,k,..., m,n,£)  completely 
characterizes  the  forms  taking  the  path i,j,k,...,Tn,n,t. 

Statement  3  A  class  with  path  a,i,j,k . m,n,t,u  is  characterized  by 

F(a,i,j,k,...,7rL,Ti,t,  cj).  If  F (a, i,j, k,... ,7n,n,t,  cj)  is  not  satisfiable  the  class  is  empty. 

We  also  have  to  deal  with  infinite  traces. 

Statement  4  Consider  two  stations  k  and  l  and  three  arbitrary  sequences  of  sta¬ 
tions  x.y.z.  If  F (x,k,l,y,k,l,z)  is  true  for  a  form  v  then  so  is  F (x,k,l,  (y,k,L)*,z). 
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That  is,  the  portion  y,k,  l  can  be  repeated  an  arbitrary  number  of  times. 

Statement  5  If  a  pair  of  stations  k,l  repeats  in  a  trace  then  the  trace  will  be 
cyclic. 


There  are  a  finite  number  N  of  stations  in  the  station  diagram.  If  a 
path  becomes  longer  than  N 2  then  a  station  will  repeat  at  least  N+ 1  times.  If  a 
station  repeats  at  least  N+l  times,  then  a  pair  of  stations  will  repeat. 

Statement  6  If  a  trace  is  longer  than  N2,  then  it  is  cyclic. 

Statement  7  If  a  trace  is  infinite,  then  it  is  cyclic. 

Consider  a  cyclic  trace.  If  its  cycle  is  longer  than  N 2  then  there  mil  be 
a  pair  of  stations  repeating  following  the  same  arguments  as  before.  Hence 
there  will  be  a  smaller  cycle. 

Station  8  If  a  trace  is  cyclic  its  cycle  has  length  at  most  N2. 

We  know  that  all  classes  have  either  finite  paths  of  length  up  to  N 2,  or 
they  are  cyclic  with  cycles  up  to  N2.  Since  the  length  of  the  class  paths  is 
bounded  it  follows  that  there  is  only  a  finite  number  of  non  empty  classes. 

Statement  9  There  is  a  finite  number  of  classes  (less  than  NnZ  )  with  finite  paths. 

Statement  10  There  is  a  finite  number  of  classes  (less  than  Nn Z)  with  infinite 
cyclic  paths. 

From  our  previous  discussion  it  follows  that  all  traces  are  predictable. 
There  is  a  simple  but  inefficient  algorithm  to  determine  all  the  classes  and  the 
characteristic  functions  of  each  class.  The  algorithm  is  as  follows 

1)  Get  all  paths  up  to  N2  which  do  not  have  cycles.  For  each  one 
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compute  the  characteristic  function. 

2)  Get  all  paths  with  cycles  up  to  Nz.  For  each  one  compute  the 
characteristic  function. 

Obviously  there  are  better  ways  to  obtain  the  classes  from  the  station 
diagram.  We  propose  an  algorithm  based  on  the  observation  that  if  a  pair  of  sta¬ 
tions  repeats  then  we  do  not  have  to  go  on.  The  algorithm  is  outlined  in  terms  of 
an  example  (Figure  la,  lb). 

First  we  identify  all  generating  stations,  i.e.,  stations  i  such  that 
F(a,i,j)  is  satisfiable  for  some  j.  Starting  with  the  generating  stations,  we  follow 
the  edges  of  the  station  diagram  in  a  tree  fashion  (Figure  2b).  For  instance,  in 
our  example  stations  1  and  2  are  generating  stations.  Each  time  we  follow  an 
edge  we  incorporate  the  appropriate  F(k,l,m)  into  the  characteristic  function. 
If  the  characteristic  function  of  a  path  becomes  false  for  all  forms  we  do  not 
continue  on  that  path.  If  a  pair  of  stations  repeats  it  is  a  potential  cyclic  path. 
For  instance,  in  our  example  at  the  double  circle  stations  2  and  3  we  have 
detected  repeating  pairs  of  stations.  We  stop  the  path  and  check  to  see  whether 
the  characteristic  function  is  satisfiable.  The  algorithm  gives  us  all  non  empty 
class  paths  with  the  characteristic  functions  of  the  classes.  In  our  example  the 
class  paths  are:  (a,  1,  2,  3,  cj),  (a,  1,  2,  4,  cj),  (a,  1,  4,  cj),  (a,  2,  3,  1,  2,  4,  cj),  (a,  2,  3,  cj) 
and  (a,  2,  4,  cj).  Using  this  information  we  outline  the  form  flow  diagram  for  one 
example  in  Figure  2a. 

Suppose  that  the  characteristic  functions  F(i,j,k)  are  boolean  func¬ 
tions  of  single  conditions  x,...,z  on  a  form’s  field  values.  The  characteristic  func¬ 
tions  of  the  paths  can  then  be  reorganized  from  a  conjunction  of  functions 
F(i,j,k )  into  a  disjunctive  form  of  boolean  conjunctions  involving  the  simple  con¬ 
ditions.  Moreover,  the  simple  conditions  can  be  grouped  according  to  form 
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fields.  In  this  way,  we  can  obtain  the  range  of  values  for  different  fields  for  which 
a  particular  path  characteristic  function  is  true.  Suppose  now  we  have  a  distri¬ 
bution  of  frequency  of  field  values  for  each  form  type.  This  information, 
together  with  the  overall  volume  of  the  forms  generated  in  each  station,  enables 
us  to  compute  an  expected  volume  for  each  class.  By  summing  up  expected 
volumes  of  appropriate  classes  we  can  obtain  the  expected  volumes  for  the 
forms  entering,  leaving  and  staying  in  each  station. 

Suppose  now  we  have  an  average  cost  per  form  to  handle  a  form  of  a 
given  type  q  that  a  particular  station  is  dispensing.  The  expected  volume 
together  with  the  average  cost  gives  us  the  load  L ±  of  that  station. 

3.  Diagram  reorganization 

Assume  that  for  a  given  form  manipulation  system,  we  have  deter¬ 
mined  the  form  classes.  We  also  know  the  form  paths  associated  with  each  class 
and  have  constructed  a  form  flow  diagram.  There  are  some  reorganizations  we 
can  make  on  the  stations  retaining  effectively  the  same  office  procedures. 

The  reorganizations  have  as  their  objective  the  simplification  of  the 
form  flow,  the  balancing  of  station  load,  or  the  optimization  of  different  cost 
functions.  Consider,  for  instance,  a  station  which  is  associated  with  a  set  of  form 
paths.  The  nodes  of  the  form  flow  diagram  for  this  station  can  be  separated  and 
each  one  associated  with  each  class.  The  station  can  be  effectively  split  in 
different  stations  with  a  new  station  associated  with  each  node  of  the  form  flow 
diagram.  Consider,  as  a  second  example,  two  stations  i,  j.  The  station  j  can  be 
merged  into  station  i  and  considered  as  one  station  under  certain  conditions. 
The  flow  diagram  is  changed  accordingly.  Station  merging  may  introduce  addi¬ 
tional  repeating  stations  in  the  paths.  If  i  and  j  were  part  of  a  common  path  the 
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new  station  i  will  repeat  in  that  path.  Notice  that  station  merging  is  essentially 
a  reclustering  of  the  nodes  of  the  form  flow  diagram.  If  the  geography  of  the 
stations  is  important,  we  should  indicate  the  position  of  the  new  cluster  and 
update  distances  between  stations  accordingly  (merging  into  i  different  than 
merging  into  j).  In  order  to  formalize  the  ideas  of  splitting  and  merging  of  sta¬ 
tions  we  need  a  notion  of  equivalence. 

Consider  a  station  diagram.  We  define  as  A(i,j,k)  the  action  in  station,/ 
taken  on  a  form  v  coming  from  station  i  and  going  to  station  fc.  The  function 
A(i,j,k)  is  a  function  on  the  values  of  the  form  fields.  We  extend  the  functions 
A  ( i,j,k )  in  a  natural  way.  For  every  station  i  including  the  source  station  a,  for 
every  station  j,k,...,m,n  and  for  every  station  t  including  the  sink  station  a. 

A(i,j,k,...,m,n,t)  =  A  {itj,k,...,m,n)  *  A(jn,n,t) 

The  function  A{i,j,k,...,m,n,t )  is  interpreted  as  the  composition  of 
actions  taken  by  the  stations  when  a  form  follows  the  path  i1j,k,...,mnn,t.  For  a 
form  originating  at  station  i  the  function  A  (at, i,j,... )  defines  the  combined  action 
on  that  form.  We  can  introduce  now  a  notion  of  equivalence  between  station 
diagrams. 

Two  station  diagrams  are  equivalent  when  every  form  v  has  the  same 
combined  action  in  each  station  diagram.  That  is,  for  every  form  v  and  an  ori¬ 
ginating  station  i  in  one  station  diagram  there  is  another  originating  station  j  in 
the  second  station  diagram,  such  that,  A  (a.i,... )  =  A  (a,/,...  )  on  the  form  v. 
Notice  that  the  two  forms  do  not  have  to  have  the  same  trace.  As  a  matter  of 
fact,  the  station  diagrams  can  have  different  structure  and  the  traces  can  be 
quite  different. 
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Consider  a  form  flow  diagram.  Consider  a  node  in  the  form  flow 
diagram  corresponding  to  a  station  k  occurrence  in  a  class  path  a,...,i,j,k,m,n,.... 
Some  of  the  activity  of  station  k  can  be  split  into  a  separate  station  p.  The 
characteristic  functions  for  station  p  are  defined  as 


F(ij,p)  -  F(i,j,k),  F{j,p,m.)  =  F{j,k,m,),  F(p,m,n)  =  F(k,7n,n) 
and  the  characteristic  functions 

F ('i.j.k),  F (j, k,  m)  and  F  (fc,m,7i) 
are  redefined  to  be  false.  The  action  functions  are  defined  as 

A  ( i,j,p  )  -  A{i,  j,  k),  A  ( j.p ,  m)  =  A  (J.k.m) 

and 


A  (p,  m,  n )  =  A  (k,m,n) 

The  net  effect  of  this  change  is  to  transfer  any  flow  which  follows  the 
path  i,j,kjTn,n  to  the  path  i j.p.rrun.  Any  other  flow  will  remain  the  same.  Notice 
that  more  than  one  class  path  can  be  changed  as  a  result.  In  addition,  there  are 
some  boundary  cases  where  ail  the  activity  is  transferred  from  station  k  to  sta¬ 
tion  p. 


We  have  defined  station  splitting  as  an  equivalence  preserving  transfor¬ 
mation  of  the  station  diagram.  Station  splitting  can  be  carried  to  the  extreme 
by  separating  all  the  nodes  of  the  form  flow  diagram  into  separate  stations. 
Cycles  in  cyclic  paths  are  separated  as  cycles  of  newly  defined  stations.  We  can 
end  up  with  a  flow  diagram  consisting  of  a  set  of  disjoint  paths  having  no  com¬ 
mon  station.  The  paths  can  be  split  even  further  by  splitting  arbitrarily  their 
generating  stations.  This  transformation,  called  class  splitting,  produces  paral¬ 
lel  paths  which  have  disjoint  stations  doing  exactly  the  same  actions.  Class  split¬ 
ting  may  seem  artificial  but  it  is  important  when  we  want  to  create  parallel 
paths  to  cut  down  on  the  load  handled  by  each  station  by  reducing  the  volumes 
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of  forms.  Class  merging  can  only  be  effected  provided  the  two  classes  have  the 
same  combined  action. 

Station  merging  can  also  be  done  as  an  equivalence  preserving 
transformation.  Consider  two  stations  k  and  l  such  that  there  is  no  station  i 
connected  in  the  same  way  to  both  k  and  l  in  the  station  diagram.  That  is  the 
stations  i  sending  forms  to  k  and  the  stations  j  sending  forms  to  l  are  disjoint. 
In  addition,  the  stations  m  receiving  forms  from  k  and  the  stations  n  receiving 
forms  from  l  are  equally  disjoint.  In  this  case  k  and  l  can  be  merged  into  one 
station  p  with  the  following  properties. 

F{ip,m)  =  F{i,k,m)  F(j,p,n)  =  F(j,l,n) 

F{i,p,n)  =  F(i,k,l,n)  Fij.p.m)  =  F(j,l,k,n ) 

A  ( i,p,m )  =  A  (i,k,m)  A  ( j,p,n )  =  F ( j . l,n) 

A  {i,p,  n )  =  A  (i,k, l, n)  A(j,p,m)  -  F(j,l,k,n) 

Notice  that  if  a  station  t  was  common  sender  (or  receiver)  to  both  k  and  l,  then 
it  will  be  impossible  to  make  a  distinction  after  merging  between  the  paths  t,k,... 
and  Notice  that  we  did  not  make  any  assumptions  about  the  direct  con¬ 

nections  between  the  merged  stations  k  and  l.  If  k  and  l  are  directly  connected, 
then  we  have  a  case  of  serial  merging  in  the  diagram.  If  k  and  l  are  not  con¬ 
nected,  then  it  is  a  merging  between  two  stations  which  are  to  a  large  extent 
independent. 

Consider  a  station  diagram.  We  obtain  the  form  flow  diagram  accord¬ 
ing  to  section  2.  We  then  reorganize  the  stations  doing  first  splitting  along  the 
class  paths  until  they  are  independent.  We  follow  with  merging  serially  all  sta¬ 
tions  along  the  same  path.  We  end  up  with  a  canonical  station  diagram  which 
has: 


1)  for  each  finite  path  class  characterized  by  the  function 
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F u)  one  station  which  is  a  generating  station,  a 
repository  station  and  has  an  action  of  A  (a.i,  j,...,Tn,n,t,  cj) 

2)  for  each  cyclic  class  path  characterized  by  the  function 
F(a,i,...,k,l(x,k,l)*)  one  station  which  is  an  originating  station  with 
action  A  (a,i,...,k,l)  and  second  station  completing  a  loop  with 
action  A  ( x,k,l ). 

This  simplified  station  diagram  is  equivalent  to  the  original  diagram.  It 
is  also  canonical  in  the  following  respect.  Any  station  diagram  equivalent  to  the 
starting  diagram  is  reduced  to  the  same  canonical  diagram  up  to  station  renam¬ 
ing  and  class  splitting  in  parallel  paths. 

In  all  our  discussion  of  station  reorganization  we  did  not  take  into 
account  constraints,  like  stations  which  cannot  move,  dependency  among  paths 
of  different  form  types,  etc.  It  should  be  clear  that  if  there  are  any  special  con¬ 
straints  the  merging  and  splitting  can  only  be  effected  when  they  are  not  violat¬ 
ing  the  constraints.  For  instance,  in  figure  3a,3b,3c,  we  outline  a  station  reor¬ 
ganization,  while  preserving  the  position  of  two  stations  because  of  coordination 
with  other  form  flows.  This  example  portrays  the  flow  of  a  loan  application.  The 
analysis  shows  that  the  actions  of  the  credit  clerk  and  processing  clerk  stations 
can  be  moved  to  the  receptionist  station. 

4.  Concluding  remarks 

In  this  paper  we  outline  some  tools  to  be  used  for  the  analysis  of  docu¬ 
ment  flow  in  terms  of  forms  in  office  systems.  We  have  outlined  a  method  for 
classifying  forms  in  different  classes  and  obtaining  the  flow  path  of  each  class. 
The  analysis  is  based  on  local  information  in  each  station  about  the  treatment  of 
forms.  The  work  is  based  on  the  assumption  that  a  form's  progress  depends 
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only  on  its  values  and  its  immediate  history.  In  addition,  we  expect  the  next 
destination  of  a  form  to  be  decided  deterministically  within  a  station.  A  form’ 9 
progress  may  be  dependent  on  another  form  reaching  a  station.  However,  we 
expect  that  the  second  form  will  eventually  appear,  and  once  it  is  there  the  pro¬ 
gress  of  the  first  form  is  determined. 

The  determinacy  constraint  could  be  eased  by  exploring  potential 
behaviour  rather  than  sure  behaviour.  Let  us  suppose  that  at  a  junction  station 
i  the  form  may  go  either  to  j  or  to  k  depending  not  on  its  contents  but  on  other 
prerogatives,  e.g.,  the  existence  of  another  form.  We  can  model  this  behaviour 
in  one  of  two  ways.  We  can  have  a  notion  of  non-determinacy  by  defining  form 
paths  as  trees  and  depicting  all  stations  to  which  a  form  may  go.  We  can  also 
have  a  probabilistic  model  by  assigning  probabilities  to  each  branch  of  the  tree. 
In  the  first  case  we  can  do  worst  case  analysis  and  in  the  second  probabilistic 
analysis  of  form  flows.  In  this  paper,  however,  we  restricted  our  attention  to 
deterministic  models. 

We  give  a  way  of  obtaining  an  indication  of  the  volume  of  forms  in  each 
class.  Based  on  that  we  can  compute  the  load  of  each  station  and  compare  it 
with  the  capacity  of  the  station.  Other  characteristics  of  the  form  traffic  can  be 
computed  if  we  assume  distances  between  stations  and  queueing  disciplines  for 
the  handling  of  the  forms  in  each  station. 

In  section  3  we  propose  some  reorganizations  of  the  station  diagram 
while  retaining  effectively  the  same  office  procedures  in  terms  of  actions  on  the 
forms.  Reorganization  steps  are  outlined  in  section  3,  namely:  station  splitting, 
station  merging,  class  splitting,  class  merging.  The  restructuring  can  have  as  its 
goal  the  optimization  of  different  cost  functions.  In  addition,  it  can  be  subject  to 


different  constraints. 


Form  Flow  Models 


(1)  Given  a  form  flow  diagram,  design  another  one  where  the  sum  of  distances 
over  all  volumes  and  paths  is  minimal  subject  to  the  constraint  of  stationary  sta¬ 
tions.  That  is,  some  stations  cannot  move. 

(2)  Given  a  form  flow  diagram,  design  another  one  where  the  load  of  each  station 
is  the  closest  to  a  set  of  station  capacities.  That  is,  the  capacity  of  a  station  is 
fixed  based  on  a  person's  capacity  for  work. 

(3)  Given  a  form  flow  diagram,  design  another  where  the  channel  capacities  of 
the  station  to  station  interaction  are  given. 

(4)  Design  a  form  flow  diagram  with  variable  station  loads  which  has  the  simplest 
structure. 

(5)  Assuming  that  certain  paths  (or  parts  of  paths)  have  no  fixed  order  of  action 
in  the  nodes  redesign  the  form  flow  diagram  to  minimize  the  distance  travelled. 

(6)  Concentrate  on  coordination  nodes  by  abstracting  away  action  nodes. 
Analysis  of  coordination  can  be  done  with  models  such  as  Petri  nets  or  other 
models  of  asynchronous  behaviour. 

(7)  Assuming  queueing  disciplines  for  each  coordination  node  optimize  the 
expected  time  of  path  traversal  on  the  basis  of  distance  and  action  weights  in 
the  stations. 

Some  of  these  problems  can  be  studied  as  network  flow  problem  [Ford 
and  Fulkerson].  The  emphasis,  however,  should  be  on  simplification  of  pro¬ 
cedures,  not  only  optimization  of  flow.  As  a  matter  of  fact,  we  expect  that  office 
procedures  with  machines  may  have  simpler  structure  than  manual  paper  sys¬ 
tems.  In  manual  systems  we  are  constrained  by  geography,  specialization,  max¬ 
imum  capacity  and  minimum  requirements  for  work,  and  cost  of  shipping  forms. 
In  restructuring  for  computerized  form  systems  we  are  liberated  from  these 
constraints.  The  emphasis  should  be,  therefore,  on  function  and  structure 
simplification  not  constraints. 
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la.  Station  Diagram 


F  Cot ,  1 , 2)  =z 
F  (3, 1 , 2) =x ' 

F  (a, 2, 3)=y 
F (1 , 2, 3) =x 
F (2,3 , 1) =x  ?y 
F  (2, 3,0))  =x 
F(l,4,a))  =  i 


F (a, 1 ,4) =z 
F (3, 1 ,4) =x 
F (a, 2,4)=y ' 

F (1 >  2, 4) =x ’ 

F  (2, 3, 4)  =x '  y 
F(2,4,a0=l 
F  (3,4,0))  =1 


Where  x,y,z  are  Boolean  conditions  and 
x  ,y  , z  their  negation. 


lb.  Characteristic  Functions 


2a.  Form  Flow  Diagram 


2b.  Paths  and  Characteristic  Functions 
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3a .  Loan  application  Station 
Diagram 
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Bureau 
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3c .  Modified  Station  Diagram 
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Abstract 


Data  translation  and  transaction  translation  are  t 
problems  that  have  to  be  solved  in  order  to  achieve 
existence  of  heterogeneous  distributed  data  bases.  In  th 
we  address  the  problem  of  transaction  translation.  The  n 
the  problem  is  explored  by  developing  direct  transla 
transactions  between  the  relational  and,  hierarchical  and 
models.  Methods  for  mapping  a  hierarchical  or  network  s 
an  equivalent  relational  schema  are  presented.  The  re 
operations  projection,  restriction,  join,  insertion,  d 
and  update  are  translated  to  equivalent  hierarchical  or 
operations. 
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1  Introduction 


Centralized  systems  have 

been  the 

major 

trend  in 

the 

computer 

field  for  a  number  of  years. 

However , 

the 

advent 

of 

computer 

network  technology  has  res 

ulted  in 

a 

growing 

trend 

towards 

developing  distributed  systems.  A  Distributed  Data  Base 
Management  System  is  loosely  defined  as  a  system  managing  a 
logically  integrated  data  base  which  is  physically  distributed 
ever  several  distinct  computing  facilities  [DEPPE  1976], 

There  are  several  ways  in  which  the  data  can  be  distributed- 
Cne  approach  assumes  that  there  exists  a  global  data  base  which 
is  distributed  in  a  controlled  fashion  to  the  different  nodes  of 
a  network  [BCTHflIE  1977].  However,  there  are  situations  where 
such  a  scenario  is  not  realistic.  The  trend  in  information 
systems  is  increasingly  towards  providing  an  integrated  view  of 
an  organization's  data.  In  any  large  organization  the  data  may 
be  distributed  over  many  departments  and  perhaps  even  over 
several  geographic  locations.  The  different  departments  or 
locations  may  have  computerized  on  an  individual  basis,  choosing 
different  DBMSs  that  met  their  particular  needs.  is  these 
organizations  move  to  integrate  their  data,  they  are  faced  with 
the  need  for  communication  and  cooperation  between  heterogeneous, 
distributed  data  bases.  We  may  be  confronted,  therefore,  with 
several  existing  data  bases  running  on  different  systems  and 
machines  that  should  be  accessed  in  a  uniform  way. 

Data  translation  and  transaction  translation  are  two  major 
problems  that  have  to  be  solved  in  order  to  achieve  the  co¬ 
existence  of  heterogeneous  data  bases.  For  an  on-line,  fast 
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response  environment,  data  translation  per  se  does  not  provide  a 
complete  solution.  The  restructuring  and  transmission  of  large 
amounts  of  data  may  not  be  economical.  A  more  promising  approach 
may  be  to  translate  transactions  from  one  DBMS  to  another,  and  to 
process  the  transactions  in  the  environment  of  the  EBMSs  where 
the  data  reside.  In  this  way,  only  transactions  and  the 
retrieved  data  need  to  be  translated  and  shipped  over  the 
retwork,  not  entire  data  bases. 

This  paper  discusses  the  translation  of  transactions  between 
the  three  basic  models  (hierarchical,  network  and  relational). 
It  does  not  deal  with  data  base  translation  and  restructuring. 
Cur  aim  is  to  determine  the  nature  of  the  transaction  translation 
problem  by  first  attempting  direct  mappings  of  operations  on 
data  bases  which  are  based  on  different  data  models.  In  this 
way,  a  user  can  access  a  global  data  base  with,  say,  relational 
commands,  while  parts  of  it  are  organized  according  to  the 
relational,  hierarchical  or  network  model. 
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2.  Framework 

We  consider  a  data  base  as  consisting  of  a  schema*  a  set  of 
states  and  a  set  cf  operations  [KLUG  1978]-  The  schema  has 
generally  two  parts,  a  naming  part  (structures)  and  a  constraint 
part.  The  constraints  (properties  of  the  data  base)  may  be 
classified  as:  explicit  (declared  in  the  schema) ,  inherent 
(expressed  by  the  structure)  and  implicit  (implied  by  other 
existing  constraints)  r BRODIE  1978].  A  data  base  state  is 
determined  by  the  values  of  a  set  of  objects.  For  instance,  a 
hierarchical  data  base  state  is  determined  by  the  data  base 
proper,  the  position  pointer,  etc. 

We  say  that  a  data  base  is  sch eaa- equivalent  to  another  data 
base,  if  there  exists  a  mapping  that  maps  the  schema  S2  of  the 
second  data  base  to  the  schema  S 1  of  the  first  data  base,  such 
that  all  constraints  (properties)  in  S2,  that  are  essential  in 
the  context  cf  the  first  data  base,  can  be  preserved  in  SI. 
There  may  be  some  properties  in  the  schema  of  the  second  data 
base  that  are  immaterial  in  the  context  of  the  first  data  base. 
For  instance,  the  set  ordering  for  a  network  schema  is  immaterial 
in  the  relational  model  (context  of  the  relational  data  base). 
3n  this  respect,  the  schema  mapping  is  not  always  reversible. 

We  say  that  a  data  base  is  operation-equivalent  to  another 
data  base,  if  it  is  schema-equivalent  and,  every  operation  on  the 
first  data  base  can  be  mapped  to  (a  series  of)  operations  on  the 
ether  data  base,  such  that  the  consistency  of  the  data  base 
states  is  preserved. 
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We  show  in  this  paper  that  for  any  hierarchical  or  network 
data  base,  we  can  generate  a  relational  data  base  which  is 
cpera tion-egui val ent  to  the  hierarchical  or  network  data  base. 
First  we  show  the  schema  equivalence,  and  then  we  map  the  basic 
relational  operations  (projection,  restriction,  join,  insertion, 
deletion,  update)  to  hierarchical  or  network  operations. 

We  define  the  network  and  hierarchical  schemas  inductively 
[VASSILIOU  1578].  The  structures  in  a  network  (hierarchical) 
schema  are  record  types  (segment  types)  and  network 
(hierarchical)  links.  Simple  inductive  rules  are  used  to 
generate  the  schemas  (e.g.  add  a  record  type,  add  a  network  link, 
etc.).  For  each  network  (hierarchical)  rule,  there  is  a  mapping 
tc  a  relational  schema  generation  rule. 

To  map  network  and  hierarchical  schemas  we  need  to  specify 
constraints  in  the  relational  schema.  All  constraints  are  in 
procedural  form.  A  basic  constraint  is  the  foreign-key 
dependency  constraint  or  simply  dependency  constraint.  Our 
notation  for  the  dependency  constraint  is: 

value  FK  (Ei)  in  E  j  dependent  on  value  K  (Ei)  in  Pi 
This  means  that  for  all  tuples  in  B j ,  the  values  of  attribute (s) 
FK(Ei)  (foreign-key  of  Bi  in  Ej)  are  a  subset  of  the  values  of 
atttribute  (s)  K(Ei)  (key  of  Ei)  in  Ei.  Because  of  the  uniqueness 
of  keys  in  a  relational  system,  the  dependency  constraint  implies 
a  triggered  delete  (if  necessary)  in  Ej  whenever  an  Pi  tuple  is 
deleted. 

Since  we  are  mapping  operations,  we  eliminate  the  need  to  be 
confined  to  a  particular  relational  data  manipulation  language. 
The  mappings  require  .  the  existence  of  null  values.  It  is 
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recognized  that  null  values  are  sometimes  very  useful, 
especially  when  they  are  given  the  "presently  unknown"  semantic 
interpretation  [CCDD  1975]  [ ZANTOLO  1978]. 

3 .  Hierarchical  Transforms 

The  hierarchical  to  relational  schema  mapping  is  based  on  the 
key-propagation  idea  which  is  very  similar  to  the  process  of 
normalization  introduced  by  Codd.  There  is  a  one  to  one 
correspondence  between  segment  types  and  relations.  In  addition, 
the  key  fields  of  higher  level  segment  types  are  propagated  to 
lower  level  segment  types. 

More  formally,  let  S  be  a  hierarchical  schema  with  k  segment 
types  and  m  hierarchical  links  (parent-child  relationships).  The 
schema  mapping  is: 

1.  For  each  root  segment  type  define  a  relation  such  that: 

(a)  each  attribute  of  the  relation  corresponds  to  a  field  of 
the  segment  type; 

(b)  the  key  of  the  relation  is  equal  to  the  key  of  the 
segment  type. 

2.  For  each  dependent  segment  type  and  the  parent-child 

relationship  in  which  it  is  a  child,  define  a  relation  such 
that : 

(a)  each  attribute  of  the  relation  corresponds  to  a  field  of 
the  segment  type  plus  the  foreign  key  of  the  relation 
based  on  the  parent  segment  type; 
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(b)  the  key  of  the  relation  is  equal  to  the  key  of  the 
segment  type  plus  the  foreign  key  of  the  relation  based 
on  the  parent  segment  type; 

(c)  the  value  of  the  foreign  key  in  the  relation  based  on 
the  child  segment  type  is  dependent  on  the  value  of  the 
key  in  the  relation  based  on  the  parent  segment  type 
(dependency  constraint) . 

Any  hierarchical  link  is  an  inherent  integrity  constraint. 
It  means  that  a  child  segment  occurrence  is  connected 
automatically  to  a  parent  segment  occurrence  and  may  not  be 
removed  unless  it  is  deLeted.  Following  the  DBTG  terminology, 
the  hierarchical  link  is  of  type:  fixed- automatic.  We  express 
this  in  the  relational  schema  with  the  foreign  key  dependency 
constraint.  An  implicit  constraint  derived  from  the  dependency 
constraint  is  the  assertion  (triggered  delete): 

Assert:  On  deletion  of  an  Fi  tuple  with  value  K(Ri)  =  v 
trigger:  delete  all  F  j  tuples  with  value  FK  (Fi)  =  v 
where  Fi,  Fj  are  based  on  the  parent  and  child  segment  types 
respectively. 

The  schema  mapping  assumes  the  notion  of  a  key  (partial- 
key)  in  the  hierarchical  model.  In  this  respect,  we  have  an  IMS- 
like  data  base. 

The  hierarchical  model  is  inherent  in  the  facilities  of  IMS 
and  SISTEM-2000.  For  the  source  system,  we  will  assume  an  IMS- 
like  record-at-a-time  navigational  language.  The  basic  commands 
are*  qet;-P€xt  and  ge  t-ne  xt-within-  pa  rent  [IMS  1975].  We  use  a 
simple  syntax  in  our  query  language  to  avoid  the  intricacies  of 
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IP'S.  Whenever  necessary,  we  will  discuss  the  consequences  of 
having  a  language  similar  tc  the  En glish- key word  query  language 
cf  SYSTEM-2000  [SYSTEM-2000  1974].  Here,  the  main  command  is 
HINT  and  there  are  features  in  the  qualification  that  allow 
downward  or  upward  hierarchical  normalization. 

We  assume  an  output  command  that  brings  into  the  user's 
workspace  specified  field  values  of  an  accessed  segment 
occurrence.  For  simplicity,  we  make  the  assumption  that  for  all 
algorithms  the  traversal  of  the  hierarchical  data  base  starts 
from  the  first  hierarchical  data  base  record. 

The  algorithms  assume  a  global  enumeration  of  the  segment 
types  in  the  hierarchical  data  base  definition  tree. 
Furthermore,  for  each  retrieval  involving  a  target-list  and/or  a 
qualification  term,  a  new  enumeration  of  the  referenced  segment 
types  according  to  their  position  in  the  hierarchy,  is  assumed. 
With  these  enumerations  we  are  able  to  present  the  algorithms  in 
a  concise  form. 

Iiojection 

Projection  of  more  than  one  attribute  of  a  relation  based  on 
a  segment  type  would  generally  reguire  a  recursive  algorithm  with 
extensive  sequential  search  of  the  data  base. 

First,  the  segment  types  which  contain  the  fields  corresponding 
tc  the  projected  attributes  are  determined.  Ey  construction  of 
the  relations  the  segment  types  must  be  in  the  same  hierarchical 
path.  Suppose  that  these  segment  types  are:  H 1, H2, . . . , Hk.  The 
segment  type  HI  is  the  one  at  the  highest  level.  Informally,  the 
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algorithm  fcr  projection  is:  For  each  highest  level  referenced 
segment  (cf  type  HI)  in  the  data  base,  retrieve  the  second 
highest-level  referenced  descendant  segments  (of  type  H2)  and 
continue  likewise  dewn  the  hierarchy  until  the  lowest  level 
referenced  descendant  segments  are  retrieved. 


algorithm  Projection 

LOOP:  get-next  HI  segment 
exit  if  none 

output  referenced  field  values 
RETRIEVE-CHILDREN  (1,k) 
continue  from  LOOP 

recursive  procedure:  RETRIEVE-CHILDREN  (i,k) 
i  :=  i+1 
exit  if  i>k 

LOO P 1 :  get- next- within- parent  Hi  segment 
exit  if  no  more  children 
output  referenced  field  values 
BETB I EVE-CHI I DREW  (i,k) 
continue  from  LOOP1 


This  algorithm  recursively  nests  qet-next-withi n- parent 
statements.  Some  hierarchical  systems  may  not  support  such  a 
feature.  For  these  systems,  we  may  need  restrictions  on  the 
maximum  path  length  that  may  be  referenced  in  a  query. 

IMS/VS  offers  a  code  option  (D-option)  that  allows  path 
calls,  i.e.  multiple  segments  in  a  hierarchical  path  are 
retrieved  in  a  single  call.  The  retrieved  segments  are 
concatenated  in  the  user's  I/O  area.  Bepeated  path  calls  and  a 
masking  of  the  unspecified  fields  would  produce  the  desired 
result  of  projection. 

Path  calls  are  allowed  in  SYSTEM-2000,  hence  projection  is 
expressed  directly.  The  command  is:  PRINT  <ob ject-list> ,  where 
object-list  is  a  list  of  field  references  or  segment  references. 


1. 


4. 


1. 

2. 

-w  m 

4. 

C 

w  • 
€. 
1. 
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Bestricticn 


We  want  to  retrieve  tuples  of  a  relation  according  to 
qualifications  on  one  or  more  attributes  of  the  relation.  The 
qualification  in  a  restriction  is  a  Boolean  (AND/OB)  on  simple 
conditions.  We  assume  that  the  qualification  can  be  partitioned 
into  separate  ti  terms  where  each  ti  applies  tc  only  one  segment 
type  Hi.  In  this  situation,  three  cases  arise.  The  first  case 
is  when  the  Boolean  operator  between  ti  terms  is  only  ANC.  That 
is,  the  qualification  is  "ti  AND  ...  AND  tk"  where  the  ti's  are  a 
series  of  simple  conditions  on  respectively  the  segment  type  Hi. 
The  algorithm  for  restriction  is  similar  to  the  one  for 
projection.  The  strategy  is  to  first  visit  the  highest  level 
segment  and  evaluate  the  condition.  Then,  if  the  evaluation 
results  in  a  true  value,  the  descendants  are  accessed  for  further 
condition  evaluation. 


Algorithm  Bestriction  (a)  (AND) 

1.  LOOP:  gei-ne^t  HI  segment  where  ti 

2.  exit  if  none 

3-  output  referenced  field  values 

li.  BETBIEVE-CHILDBEN  (1,k) 

5.  continue  from  LOOP 

1.  recursive- procedure :  FETF. IEVE-CHILDBEN  (i,k) 

2.  i  :=  i+1 

2.  exit  if  i>k 

4.  LCOEl:  get- next- within- parent  Hi  segment  where  ti 

5.  exit  if  no  more  children 

6.  output  referenced  field  values 

1.  BETBIEVE-CHIIDBEN  (i,k) 

8.  continue  from  LOOP1 


The  second  case  is  when  the  Boolean  operator  between  ti  terms 
is  only  OB.  In  this  case  it  is  necessary  to  evaluate  the 
condition  on  each  Hi  until  one  that  is  true  is  found.  When  a 
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true  condition  is  found,  then  all  descendants  gualify  regardless 
cf  the  truth  of  their  ti's. 


algorithm  Eestriction  (b)  (OB) 

1.  ICCP:  get-next  HI  segment 

2.  exit  if  none 

3.  continue  from  7  if  tl  false 

4.  RETRIEVE-UNQUALIFIED  (1,k) 

5.  output  referenced  field  values 

6.  continue  from  LOOP 

7.  RETRIEVE-QUALIFIED  (Irk) 

8.  continue  from  LOOP  if  no  Hk  segment  qualified 

S.  output  referenced  field  values 

10.  continue  from  LOOP 

1.  r ecursi ve-pr ocedu re  RETRIEVE-QUALIFIED  (i,k) 

2.  1  :=  i+1 

3.  exit  if  i>k 

4.  LOOP1:  get- next- within- parent  Hi  segment 

5.  exit  if  no  more  children 

6.  continue  from  10  if  ti  false 

7.  RETRIEVE-UNQUALIFIED  (i,k) 

6.  output  referenced  field  values 

5.  continue  from  L00P1 

1C.  RETRIEVE -QUALIFIED  (i,k) 

11.  continue  from  LOOP1  if  no  Hk  segment  qualified 

12.  output  referenced  field  values 

13.  continue  from  LOOP1 

The  procedure  RETRIEVE-UNQUALIFIED  is  identical  to  the 

procedure  RETRIEVE-CHILDREN  that  we  presented  in  the  projection 

algorithm. 

The  final  case  arises  when  ANDs  and  CRs  are  mixed  between  ti 
terms.  It  is  then  necessary  to  know  whether  the  Boolean  is  AND 
cr  OR  and  to  treat  each  case  individually.  Because  of  its 
complexity  we  do  not  present  the  algorithm  for  this  case. 

If  at  the  source  system  we  have  the  versatile  language  of 
SESTEM-2000 ,  restriction  is  expressed  very  easily.  We  can  use 
the  WHERE  clause  with  Bocleans  which  combines  several  conditions 
in  the  same  hierarchical  path  by  carrying  out  an  upward 
normalisation  on  the  selected  segments. 
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Jcin 

We  will  give  algorithms  that  translate  natural  joins  between 
two  relations  Bi  and  Bk.  The  Booleans  between  the  join  terms 
will  he  restricted  to  AMD's,  since  the  appearance  of  an  OB 
connective  will  generally  require  a  sequential  search.  For 
simplicity,  assume  that  the  target  list  may  include  domains  from 
Ei  and  Hk  only.  We  consider  two  cases  for  our  join  algorithm: 

Case  (a)  Eoth  Ei  and  Ek  appear  in  the  same  branch  of  the 
hierarchical  definition  tree. 

In  this  case  we  have  a  branch  Hr ,. . . ,Hi,. . . ,Hk  where: 
0=level  (Hr) <level  (Ei) <level (Hk) 

By  construction  of  the  relations  we  have: 

K  (Ei)  subset  of  K(Bk) 

We  call  a  key- join-term  i  he  join  of  the  whole  key  in  Bi  with  the 
equivalent  part  in  Bk. 

The  key- join-term  may  or  may  not  be  included  in  the  join 
qualification  by  the  user.  There  are,  of  course,  different 
semantics  in  each  case.  Tn  addition,  the  processing  cf  the  query 
is  drastically  different.  The  hierarchical  organization  of  the 
data  base  is  geared  very  well  for  the  case  where  the  key-join- 
term  is  included.  In  such  a  case  we  join  a  segment  occurrence 
with  all  its  descendants.  Ey  contrast,  the  absence  of  the  key- 
jcin-term  leads  to  an  extremely  expensive  algorithm  for  the  join. 
We  will  give  algorithms  fcr  both  cases. 

When  the  key- join- term  is  included  in  the  qualification,  the 
translation  algorithm  is  similar  to  those  given  for  projection 
and  restriction.  Because  of  the  propagation  of  keys,  we  know 
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that  all  descendants  will  qualify  according  to  the  hey- join-term. 
In  addition,  because  of  the  uniqueness  of  keys  only  the 
descendants  of  a  segment  can  be  joined  to  a  given  parent. 
Therefore,  we  sequentially  retrieve  all  higher-level  segments  and 
join  each  of  them  with  its  descendants  (according  to  the  join 
terms  that  are  not  part  of  the  ke y- join-term) .  Suppose  that  we 
have  a  join  between  Hi  and  Hk  with  n  join  terms  in  addition  to 
the  key- join- term . 

Algorithm  Join  (a)  (key- join-term  included) 

1.  get-next  Hi  segment 

2.  exit  if  none 

3.  set  Vj  =  value  Fij  ,  j=1,2,...,ra  (for  join  terms) 

4 .  LOCP :  get-next- within- parent  Hk  segment  where 

(Fk 1  =  VI)  AND  ...  AND  (Fkm  =  Vm) 

5.  continue  from  1  if  no  more  children 

6.  output  referenced  field  values 

7.  continue  from  LOOP 

Suppose  now,  that  the  key- join- ter m  is  net  included  in  the 
qualification.  If  the  join  is  between  Hi  and  Hk  we  basically 
need  to  sequentially  retrieve  each  Hi  segment  and  join  it  with 
all  Hk  segments. 


Algorithm  Join  (a)  (key- join-term  not  included) 


1.  set-next  Hi  segment 

2.  exit  if  none 

3.  set  Vj  =  value  Fij,  j=1,2,...,m  (for  join  terms) 

4.  reset  currency  to  start  of  data  base 

5.  LOOP:  get-next  Hk  segment  where 

(Fk1  =  V  1)  AND  ...  AND  (Fkm=Vm) 

€.  continue  from  9  if  none 

7.  SSiESt  referenced  field  values 

6.  continue  from  LOOP 

9.  reset  currency  to  last  Hi  segment 
1C.  continue  from  1 
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W €  note  that  suitable  mechanisms  must  exist  for  setting  and 
resetting  currency  indicators  as  reguired.  We  do  not  show  these 
operations  in  detail  in  our  algorithm. 

Notice  that  the  two  previous  algorithms  are  for  extreme 
cases.  In  the  first  extreme  we  have  all  keys  of  the  segment 
types  in  the  branch  appearing  in  the  qualification,  in  the  second 
we  have  none.  Different  algorithms  are  reguired  for  intermediate 
cases.  For  instance,  if  the  eguality  of  the  root-key  is  included 
in  the  qualification,  then  we  only  join  segments  that  are  in  the 
same  data  base  record. 

The  last  algorithm  can  also  service  the  case  where  we  have  Hi 
and  Hk  appearing  in  two  different  definition  trees. 

Case  (b)  Hi  and  Hk  appear  in  different  branches  of  the  same 

definition  tree. 

Let  Hr , . . . , Hin t , . . . , Hi  and  Hr , . . . , Hint , . . . , Hk  be  the  two 
tranches.  By  construction,  both  Hi  and  F.k  will  have  the  keys  of 
Er,...,Hint  as  common  attributes  (part  of  their  relational  key). 
If  the  join  of  these  common  attributes  (com men- join -term)  does 
not  appear  in  the  qualification,  then  we  use  sequential  search 
(like  algorithm  Join  (a)  (key- join- term  not  included)),  otherwise 
we  join  segments  that  appear  in  the  same  hierarchical  sub-tree 
(i.e.  ,  are  descendants  of  the  same  Hint  segment). 
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algorithm  Join  (b)  {common- join- term  included) 


1.  get-next  Hint  segment 

2.  exit  if  none 

2.  LOOP  1 :  get-next-within-parent  Hi  segment 

4.  continue  from  L00P2  if  no  more  children 

5.  place  in  buffer  the  retrieved  segment 

6.  continue  from  L00P1 

7.  L00P2:  get-next-within-parent  Hk  segment 

8.  continue  from  1  if  no  more  children 

9.  join  the  retrieved  segment  (evaluate  join  terms) 

with  all  buffered  Hi  segments 
1C.  output  referenced  field  values 

11.  continue  from  I00P2 


We  buffer  the  Hi  segments  to  save  access  time.  In  this  way 
all  Hk  segments  need  to  be  retrieved  only  cnce  for  each  Hint 
segment.  Algorithms  for  intermediate  cases,  where  the  equality 
cf  only  part  of  the  common  attributes  is  included,  are  not 
presented  here.  These  algorithms  depend  on  which  part  of  the 
common- join-term  is  present. 


Insertion 


The  insert  operation  in  a  relational  system  is  conceptually 
simple.  Tuples  are  specified  and  inserted  directly  in  a 
relation.  There  is  no  order  between  the  tuples  in  a  relation. 

Inserting  a  segment  occurrence  in  a  hierarchical  data  base  is 
a  more  complicated  operation.  It  usually  involves  a  search  for 
the  appropriate  spot  where  the  new  segment  occurrence  is  to  be 
placed.  The  search,  in  an  IMS-like  language,  is  based  on  the  key 
values  of  anscestor  segments.  A  hierarchical  path  is  established 
and  the  segment  occurrence  is  inserted  at  the  end  of  the  path. 

By  construction  of  the  relations,  the  hierarchical  path  for 
insertion  is  implicitly  provided  (propagation  of  keys) .  We 
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assume  that  in  the  hierarchical  system  there  are  no  duplicates 
under  the  same  parent. 

Let  H1,H2,...,Hk  he  segments  in  the  same  hierarchical  path, 
and  Hl,B2,...,Bk  he  the  corresponding  relations.  For  simplicity 
of  notation,  let  v1,v2,...,vn  be  the  values  for  attributes 
corresponding  to  non-key  fields  in  Hk  and,  let  71,  V2,...,  Vk  be 
the  values  of  the  keys  in  Bk  (including  foreign  keys).  Suppose 
we  insert  the  tuple  (vl ,.  . .  ,vn,V1  ,. .. ,Vk)  in  the  relation  Bk. 
Bnless  all  V  values  exist  in  a  hierarchical  path,  the  insertion 
will  te  invalid.  The  reason  is  the  presence  of  the  dependency 
constraints  for  the  relations  R2,E3,...,Ek  in  the  relational 
schema.  Bence,  the  algorithm  for  insertion  is: 


Algorithm  Insertion 


I.  set  i=r 

2-  continue  from  17  if  i=k  (Hk  is  a  root) 

3.  continue  from  7  if  i  4  k-1  (Hk  is  not  in  second  level) 

4 .  g et-h  old-next  Hi  segment  where  (K  (Hi)  =  Vi) 

3.  exit  (with  rejection  message)  i.£  none 

6.  continue  from  17 

7.  get-next  Hi  segment  where  (K  (Hi)  =  Vi)  (select  the  root) 

8.  exit  (with  rejection  message)  if  none 

9.  set  i=i+1 

10.  LCCP :  continue  from  15  if  i=k-1 

II.  get-next- wit hi n-parent  Hi  segment  where  (K  (Hi)  =  V i) 

12.  exit  (with  rejection  message)  if  do  more  children 

13.  set”i=i+1 

14.  continue  from  LOOP 

15.  q et-h old-next -wit hi n-parent  Hi  segment  where  (K  (Hi)  =  Vi) 

16.  exit  (with  rejection  message)  if  no  more  children 

17.  field-values  insert  Hk  segment 


In  SVSTEM-2000  we  can  translate  the  insertion  with  one  form 


of  the  INSERT  TBEE  command. 
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Deletion 

Assuming  that  we  have  located  the  occurrence  to  he  deleted, 
the  translation  of  the  relational  deletion  (delete  a  tuple  from 
I)  is  trivial: 

delete  H  segment 
where  IS  is  based  on  H. 

Note  that  the  hierarchical  deletion  cf  H  triqgers  the 
deletion  of  all  Qescendents  of  H.  However,  this  is  expected  by 
the  relational  user  because  of  the  dependency  constraints  for ’the 
relations . 

In  SYSTEM-2000  not  all  deletions  are  triggered-  We  defer 
discussion  cf  such  deletions  until  the  network  section-  The 
dependency  constraint  between  two  hierarchically  linked  segments 
in  a  SYSTEM-2000  data  base  is  weaker.  That  is,  the  foreign  key 
aay  have  a  null  value. 

Update 

Updates  of  key  values  in  a  relational  data  base  are  usually 
not  allowed.  Consequently,  the  translation  of  a  relational 
update  on  non-key  attributes  is  trivial. 

replace  field-value  in  H  segment  with  given  value 
It  is  assumed  that  the  segment  H  has  been  successfully 


retrieved . 
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4 .  Network  Transforms 

The  mapping  from  a  network  to  a  relational  schema  is  tased  on 
a  one  to  one  correspondence  between  data  items  and  attributes  and 
between  record  types  and  relations.  For  record  types  that  are 
members  in  a  network  link  (set  type),  the  corresponding  relation 
has  additional  attributes.  These  additional  attributes 
correspond  to  the  data  items  that  constitute  the  key  of  the  owner 
record  type  in  the  network  link.  They  express  the  relationship 
implied  frem  the  set  structure.  Network  links  differ  from 
hierarchical  links  in  that  they  have  flexible  membership 
requirements.  Set  membership  may  be  either  automatic  or  manual. 

In  addition,  the  connection  may  be  optional,  mandatory  or  fixed. 

Mere  formally,  let  S  be  a  network  schema  with  k  record  types 
and  m  network  links  (DBTG-sets).  The  schema  mapping  is: 

1.  Tor  each  record  type  define  a  relation  such  that: 

(a)  each  attribute  of  the  relation  corresponds  to  a  data- 
item  of  the  record  type; 

(b)  if  the  record  type  has  a  key  then  the  key  of  the 
relation  is  egual  to  the  key  of  the  record  type, 
otherwise  the  key  of  the  relation  is  equal  to  the 
database  key  of  the  record  type,  which  appears  as  an 
explicit  attribute  of  the  relation. 

2.  For  each  network  link  (set  type)  define  relational  integrity 

constraints  and  change  the  existing  relations  such  that: 

(a)  the  key  cf  an  owner  record  type  appears  as  a  foreign  key 
of  all  of  its  member  record  types  (add  the  necessary 
attributes  where  required)  ; 
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(t)  one  of  the  following  sets  of  constraints  applies 
depending  on  the  type  of  set  membership  (Ei  and  Ej  are 
the  relations  corresponding  to  the  owner  and  member 
record  types  respectively) : 


fixg 3-automatic 

Cl  :=  value  FK(Bi)  in  Ej  dependent  on  value  K  (Ei)  in  Ei 
C2  :=  Assert; 

On  update  of  an  Ej  tuple 
viljjfi  FK(Ri)  in  Rj  n^v^r 


Cl  ;=  value  FK(Bi)  in  Ej  dependent  on  value  K  (E  i)  in  Ei  0£  nu  11 
C2  :=  Assent; 

On  deletion  of  an  Ei  tuple  with  value  K  (Ei)  =  v 
trigger;  delete  all  Ej  tuples  wi th  value  FK(Ri)=v 
C3  ;=  Assgpt: 

On  update  of  an  Bj  tuple 

value  FK(Bi)  in  Bj  not  changed .  unless  null 


mandatory-automatic 

Cl  :=  valuq  FK(Bi)  in  Bj  dependent  on  value  K  (Ei)  iji  Ei 


lanialeri-ianual 

C 1  ;=  value  FK (Ei)  in  Ej  dependent  on  value  K(Ri)  in  Ei  0£  null 
C2  :=  Assert; 

Qn  deletion  of  an  Ei  tuple  with  value  K (Ei)  =  v 
trigger:  delete  all  Bj  tuples  wi th  value  FK(Ri)  =  v 
c 3  :=  Assert: 

On  Update  of  cj.n  Ej  tuple 

value  FK  (Ri)  in  R j  never  changed  to  null 


optional-automatic 

C 1  ;=  value  FK  (Ei)  in  Ej  dependent  on  value  K  (Ri)  in  Ei  or  null 
C 2  :=  Assert: 

On  deletion  of  an  Ei  tuple  witJi  value  K  (Ei)  =  7 
trigger:  update  all  Ej  tuples  with  value  FK  (Ei)  = 
to  FK^Ei)  <-  null 

C 3  : -  Assert: 

On  insertion  of  an  Rj  tuple 
value  FK(Ei)  in  Ej  not  null 


v 


Ill 
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optional- manual 

Cl  :=  value  FK(Ei)  in  Bj  dependent  on  valug  K  (Bi)  in  Ei  or  null 
CZ  :=  Assert: 

Cn  deletion  of  an  Bi  tuple  wi£h  yajfle  K(Bi)  =  v 
Jt£i93er:  update  ail  Bj  tuples  with  value  FK  (Bi)  =  v 
to  FK  (Ei)  <-  null 

According  to  the  latest  DELC  proposals  [MANOLA  1976]  the 
assumption  that  all  record  types  have  user-specified  unique 
identifiers  (keys)  is  justified ,  but  it  may  be  very  restrictive 
for  present  DBTG-based  systems.  In  the  case  where  a  record  type 
dees  not  have  an  explicit  unique  identifier,  the  mapping 
introduces  in  the  corresponding  relation  an  identifier  that  takes 
the  value  of  the  database  key.  This  has  some  bad  semantic 
consequences.  For  instance,  on  an  insertion,  a  value  must  be 
specified  for  the  database  key  by  the  user.  We  can  get  around 
this  situation  by  taking  advantage  of  the  inter-record 
relationships,  expressed  by  the  network  link  structure.  The 
conditions  are  that  the  record  type  which  does  not  have  a  key 
must  be  a  member  in  more  than  one  set  and  all  set  memberships 
must  be  specified  as  fixed-  or  mandatory-automatic.  In  such  a 
case  the  corresponding  relation  could  have  as  key  the  combination 
of  all  foreign  keys,  thus  eliminating  the  need  for  use  of  a 
database  key. 

The  network  language  will  be  loosely  based  on  the  language  of 
I  IMS  which  is  a  subset  of  the  DBTG  specif ications  [ IDMS  1975]. 
For  simplicity  of  illustration,  >‘e  will  assume  that,  in  addition 
tc  the  different  forms  of  the  find  command,  we  have  a  locate 
command.  This  command  is  the  combination  of  a  find  command  and 
the  programming  language  statements  for  evaluation  of  a 
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qualification  for  the  "found**  record.  The  qualification  is  a 
tcolean  combination  of  simple  conditions  on  data- item  values.  In 
addition,  the  £et  command  is  assumed  to  allow  masking  of  some 
data-item  values  that  are  transferred  into  the  workspace.  We  do 
net  distinguish  here  betveer  the  area  concept  and  the  entire  data 
tase. 

Projection 

We  will  present  the  general  algorithm  for  projection 
translation  where  all  attributes  in  a  relation  E,  which 

corresponds  to  a  record  type  N,  are  projected.  Assuming  that  in 

general  N  will  participate  as  a  member  in  several  sets,  we 
sequentially  retrieve  all  N  records  and  for  each  of  them  we  also 

retrieve  all  owners  in  the  set  types  in  which  the  N  records 

participate  as  members. 

algorithm  Projection 

1-  find  first  H  record 

2.  jjet  Dj  values,  j=1,2,...,n  /*  data  item  values  */ 

3.  for  eacfc  network  link  L  that  N  participates  as  member 

owner  record  of  I  set 
.SJii  key  value  of  owner  record 
*4.  find  next  N  record 

5.  exit  jj  none 

6 .  continue  from  2 

If  an  N  record  occurrence  is  not  currently  a  member  in  a 
set,  the  null  value  is  output  for  the  foreign  key.  There  may  be 
ether  versions  of  this  algorithm  depending  on  which  attribute  is 
projected.  For  instance,  if  a  foreign  key  is  projected,  it  may 
le  better  to  access  all  owner  records  sequentially  and  then,  if 
they  have  an  N  record  as  a  member  in  the  set  they  own. 


output 
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their  key  values.  In  general,  even  though  the  projection  and  its 
translation  are  conceptually  very  simple,  the  network  operations 
may  prove  to  he  very  expensive.  The  main  reason  is  that,  for  at 
least  one  record  type,  all  occurrences  in  the  data  base  must  be 
accessed  sequentially. 

If stricti on 

Basically,  two  strategies  may  be  used  in  the  translation  of 
the  restriction  operation.  In  the  first  strategy,  the  record 
type  N  (corresponding  tc  P)  is  used  as  an  "anchor"  record  type. 
Ir  the  se'cond  strategy  the  record  type  which  is  the  owner  in  a 
network  link  where  N  participates  as  a  member  is  the  anchor 
record  type.  It  may  not  always  be  clear  which  strategy  is  the 
"best"  ter  a  particular  situation,  but,  as  we  will  show,  there 
arc  seme  clear-cut  cases. 

icr  an  illustration  of  the  above  statements,  consider  a 
network  schema  with  two  record  types  Fi  and  Nk,  a  network  link 
lik  ard  the  corresponding  relational  schema.  Assume  that  we  have 
a  restriction  with  the  qualification  term  "ti  AND  tk",  where  ti 
is  a  simple  condition  on  the  foreign  key  in  Rk  and  tk  is  a 
qualification  on  any  ether  attributes  of  Ek.  Both  of  the 
fcllowing  algorithms  provide  correct  translations  of  this  guery. 
In  the  first  algorithm  we  use  the  member  as  the  anchor  record  and 
it  the  second  cne  we  use  the  owner. 


m  ns  <n  xr  1,1  m  _»  ln»  m  j  m  in  c  iii  m 
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Algorithm  Restriction  (a) 

1 .  locate  first  TSk  record  where  tk 

•  exit  if  none 

.  LCCP:  locate  owner  record  of  lik  set  where  ti 

•  continue  from  6  if  none 

•  get  referenced  data  item  values 

•  locate  next  Mk  record  where  tk 

•  exit  if  none 

•  continue  from  LOOP3 

lcjcrithm  Restriction  (b) 

.  locate  first  Hi  record  where  ti 
.  exit  if  none 

.  LCCP:  locate  next  Nk  record  of  lik  set  where  tk 
continue  from  7  if  none 
get  referenced  data  item  values 

•  continue  from  LCOE 

•  locate  next  Ni  record  where  ti 

•  exit  if  none 

.  continue  from  LOOP 


Clearly,  if  ti  is  of  the  form:  (FK(Ri)=v)  it  may  be  better  to 
use  the  second  algorithm.  On  the  other  hand,  if  tk  is  of  the 
form:  (K(Rk)=v1)  and  ti  is  cf  the  form:  (FF  (Ri)  *v2)  ,  it  is  very 
likely  that  the  first  algorithm  is  less  expensive- 

Qualif icaticns  with  OR  Booleans  complicate  the  translation 
algorithms.  Consider  again  the  previous  simple  network  and 
relational  schemas.  Assume  that  we  have  the  restriction  on  Rk 
with  the  qualification  term  "ti  OR  tk". 


<t\  t  n  £r  u»  k  » 
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Algorithm  Restriction  (c) 


I.  find  first  Nk  record 
exit  ij  none 

Lcci:  Continue  from  7  if  tk  is  false  for  flk 
find  owner  record  in  lik  set  (if  any) 
get  referenced  data  item  values, 
continue  from  10 

locate  owner  record  in  lik  set  wh eye  ti 

6.  continue  from  10  if  none  (ti  is  false) 

S.  get  referenced  data  item  values 

1C.  find  next  Nk  record 

II.  exit  if  none 

12.  continue  from  T.COP 


We  have  considered  restrictions  on  relations  that  involve 
crly  one  foreign  key.  The  general  case,  where  many  foreign  keys 
appear,  is  similar.  Suppose  that  we  perforin  a  restriction  on  R. 
We  retrieve  sequentially  all  E  records  in  the  data  base.  For 
each  of  the  retrieved  records,  we  access  the  owners  in  every  set 
they  participate  as  members  collecting  all  key  values.  We  may 
new  evaluate  the  qualification  and  decide  whether  to  keep  the 
retrieved  values. 


Join 


In  the  relational  model  a  join  operation  is  allowed  between 
tvic  relations  if  the  joined  attributes  are  compatible.  Eecause 
cf  the  freedom  that  the  user  has  in  forming  joins,  he  may  form 
jcins  which  are  net  meaningful.  Behind  any  meaningful  join  there 
is  an  inherent  relationship.  According  to  current  specification 
cf  the  network  model  (DBTG) ,  the  only  inter-record  relationships 
are  the  one's  that  are  explicitly  expressed  with  a  set 
declaration.  The  network  DML  is  designed  around  the  set  concept 
sc  that  it  can  take  advantage  of  the  underlying  relationships. 
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Hence,  the  only  relational  joins  whose  translation  could  be 
efficient  (where  the  network  DML  commands  can  take  advantage  of 
tie  set  construct),  are  those  that  involve  the  foreign  key,  i.e. 
the  qualification  includes  the  term:  "Bk.FK(Ei)  =  Hi .  K  (F i) In 
this  case  we  join  only  the  records  that  are  in  the  same  set 
occurrence  (owner  with  its  members) . 


Algorithm  join  (on  foreign  key) 


1. 
t . 


4. 


6. 

7. 

6. 

S. 

1C. 


find  first  Ni  record 
exit  if  none 

IcCPl:  get  referenced  data  item  values 

L00P2:  find  next  Nk  record  in  Lik  set 
continge  from  8  if  none 
get  referenced  data  item  values 
continue  from  I00P2 
fiipd  next  Ni  record 
exit  if  none 
continue  from  I00P1 


Every  other  join  that  involves  two  or  more  record 
reguire  sequential  searches  in  the  whole  data  base, 
luffer  and  complicated  programming  language  statements 


types  will 
A  large 
to  perform 


tie  join  will  he  necessary. 


Insertion 


Attribute  values  are 

specified  for  a  tuple 

to  be 

inserted  in 

a  relation 

Bk.  We  denote 

by  vl , v2, . . . , vn 

the 

values  for 

attributes 

corresponding 

to 

fields  in  Nk  and 

with 

VI, V2,.. . , Vm 

the  values  of  the  foreign  keys  in  Fk. 
The  constraints  for  insertion  are: 

1.  The  unique-key  constraint. 

2.  The  dependency  constraint. 


in  c)  cr  ui 
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Vi  4  null  for  every  i  such  that  FK(Bi)  is  a  foreign  key  in  Fk 
and  Fi  corresponds  to  Ei  and.  Hi  is  the  owner  in  a  set  that 
has  Ilk  as  member  with  automatic  set  membership  specification. 


We  give  now  the  algorithm  for  insertion. 


Sl^or it hm  Insertion 


1-  locate  all  owner  record  types  in  the  network  links  where 

membership  of  Ilk  is  automatic,  using  the  values  of  the 
foreign  keys  in  Rk ,  i.e.  establish  current  set 

occurrences  (if  any) . 

locate  all  owner  record  types  in  the  network  links  where 
membership  of  Ek  is  manual  and  there  exist  non-null 
values  for  the  foreign  keys  in  Ek,  i.e.  establish 
current  set  occurrences  for  some  of  the  manual  network 
links  (if  any)  . 

Establish  (programming  language  statements)  contents  ot 
all  Nk  record  data  items  in  working  storage 
(vl, v2,. .. , vn)  . 

Store  Ek  record 

M  record  into  all  manual  sets  that  have  been 
established  as  current  in  2  {if  any)  . 

If  Ek  never  participates  as  a  member  in  a  link,  only  steps  3 

and  4  of  the  algorithm  will  apply.  It  can  easily  be  observed 

that,  if  the  relational  constraints  for  insertion  are  not 

violated,  then  the  algorithm  can  be  applied  in  the  network  schema 

correctly . 


Deletion 

A  simple  delete  only  statement  in  the  network  system 
corresponds  to  the  relational  delete  on  the  given  relational 
schema.  Note  that  the  delete  E  record  only  statement  has  the 
following  properties: 

pi.  Femoves  an  N  record  from  all  set  occurrences  in  which  it 
participates  as  a  member. 
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p2.  Removes  tut  does  net  delete  all  optional  members,  for  each 
set  where  N  participates  as  an  owner. 
p3.  Deletes  all  fixed  and  mandatory  members  for  each  set  where  N 
participates  as  an  owner-  If  any  of  the  deleted  fixed  or 
mandatory  members  is  itself  the  owner  of  any  set  occurrences, 
then  the  DELETE  statement  is  executed  on  such  a  record  as  if 
it  was  the  object  record  (N  )  of  a  delete  only  statement. 
Thus  the  process  continues  down  the  structure  (triggered) - 

There  is  an  analogy  with  the  expectations  of  the  relational 
user  when  he  deletes  a  tuple  of  a  relation  R. 

pi*.  When  a  tuple  is  deleted,  all  attribute  values  are  deleted, 
including  the  foreign  keys  (removal  of  N  from  all  sets  as  a 
member) . 

p2*.  For  optional  membership  in  sets,  where  N  is  the  owner,  we 
have  in  the  relational  schema  an  assertion  that  foreign  key 
values  in  R  are  updated  to  pull  (optional  members  are 
removed) . 

p  2  * .  For  mandatory  or  fixed  membership  in  sets,  where  N  is  the 
owner,  we  have  an  assertion  in  the  relational  schema  that  the 
deletion  is  triggered. 

Update 

Depending  on  the  particular  attribute  that  we  want  to  update, 
a  relational  update  command  will  be  translated  to  a  simple  modify 
command  or  to  a  remove  command  cr  to  an  insert  command  or  a 
combination  of  remove  and  insert  commands. 
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Suppose  we  want  to  update  the  value  of  an  attribute  A  in  the 
relation  F  with  the  value  V.  Basically,  we  consider  two  cases. 
In  the  first  case  A  corresponds  to  a  data  item  in  the 

corresponding  record  type  N  and  we  need  a  modify  network 
command.  In  the  second  case  A  is  a  foreign  key.  This 

translation  to  network  operations  does  not  include  changes  of 
physical  values  but  removals  and/or  insertions  in  set 

occurrences . 


A  Igor it hm  Update 


1. -  if  A€  {A  1 ,  A.  2 , .  .  .  ,An}  /*  A  is  a  non-foreign  key  attribute  */ 

then  if  A=K  (F) 

then  drop  the  update 
else  do 

establish  V  in  working  storage 

modify  N  record 

end 

2. -  else  /*  A  is  one  of  the  foreign  keys  */ 

easel:  (fixed-automatic  membership) 
drop  the  update 

case  2 :  ixed-manual  membership) 

*If'v=null 

then  drop  the  update 
else  if  va lue  A  #  null 

then  drop  the  update 
else  do 

locate  new  owner  for  IT 
(using  V) 

insert  N  into  set 

case2:  (mandatory-automatic  membership) 
if  V=null 

then  drop  the  update 
else  do 

locate  old  owner  of  N 
locate  new  owner  for  N 
(using  V) 
switch  sets 
end 

case4;  (mandatory-manual  membership) 
if  V-nu 1 1 

then  drop  the  update 
else  if  value  A  =  null 
then  do 

locate  new  owner  for  N 
(using  V) 
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insert  N  into  new  set 
end 

Sise  go 

lpcate  old  owner  of  N 
locate  new  owner  for  N 
(using  V) 
switch  sets 
end 

case 5:  (optional-automatic  or  manual) 
if  V=null 
then  do 

locate  old  owner  of  N 
remove  N  from  set 
end 

else  do 

locate  old  owner  of  N 
(if  any) 

remove  N  from  set 
locate  new  owner  for  N 
(using  V) 

insert  N  into  set 
end 


The  easiest  way  to  deal  with  composite  keys  is  to  treat  them 
as  a  whole-  For  instance  a  foreign  composite  key  in  a  relation 
is  updated  only  if  all  attributes  in  the  key  are  updated-  Ir. 
this  way,  we  can  avoid  situations  where  a  composite  foreign  key 
has  seme  attributes  with  null  values  and  others  with  non-null 
values. 


5-  Concluding  re  marks 


This  paper  presents  the  mapping  of  operations  from  relational 
tc  network  and  hierarchical  data  bases.  A  general  framework  for 
transforming  schemas  and  mapping  operations  is  outlined-  vJe  do 
ret  shew  only  small  examples  of  how  a  particular  source  query  is 
translated  to  a  target  guery.  We  present  algorithms  which  may  be 
used  for  the  translation  of  any  guery  types.  The  algorithms 
presented  are  by  no  means  claimed  to  be  complete  or  optimal. 
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There  is  a  strong  emphasis  on  the  concept  cf  constraints  on  the 
data  base  and  their  preservation  in  a  schema  transform. 

From  these  direct  mappings  between  the  basic  data  models  we 
may  draw  the  following  conclusions.  The  model  to  be  used  for  the 
expression  of  the  mappings  must  be  very  powerful  in  its  treatment 
cf  constraints.  We  had  to  extend  the  relational  model  for  the 
description  of  just:  the  inherent  constraints  of  the 
hierarchical/network  structure.  In  addition,  it  seems  that  null 
values  will  inevitably  appear  in  the  mappings,  hence  they  must  be 
supported.  A  very  hard  but  also  very  important  problem  is  the 
optimization  cf  the  transaction  translation  algorithms. 

Our  direct  mappings  cf  operations  between  DBMSs  is  the  first 
step  in  determining  the  nature  of  the  interface  problem  for 
transaction  translation  among  heterogeneous  DBMSs.  Because 
organizations  have  a  high  investment  in  current  DBMS  software, 
application  programs  and  data  storage,  we  do  not  foresee  a 
wholesale  conversion  to  a  common  DBMS  for  distributed 
applications.  Instead,  organizations  will  allow  local 
installations  to  retain  their  data  model  and  data  language. 
However,  at  the  same  time,  they  would  also  like  to  permit  some 
cooperation  with  ether  data  bases  for  purposes  of  data 
integration  and  sharing.  We  believe  that  transaction  translation 
will  provide  the  facilities  to  support  the  integration  of 
heterogeneous  distributed  DBMSs. 
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Abstract 


We  start  with  a  very  brief  description  of  the  many-valued 
lcgic  approach  to  the  formal  treatment  of  null  values  in  data 
bases  and  show  some  problems  that  are  encountered.  In  the  second 
part  of  the  paper  we  present  our  approach,  based  on  the 
Eenotational  Semantics  Theory.  An  informal  introduction  to  the 
theory  is  given  and  then  the  relational  model  is  described  in 
terms  of  the  theory..  Query  evaluations  are  defined  as  continuous 
functions  and  several  examples  are  presented.  The  formalization 
in  the  framework  of  renotational  Semantics  allows  for  a  better 
understanding  of  the  semantic  problems  with  null  values.  It  also 
gives  flexibility  for  alternatives  in  acceptable  semantic 
interpretations.  We  conclude  with  a  presentation  of  an  algorithm 
for  the  evaluation  of  simple  queries.  This  algorithm  uses 
symbolic  manipulations  and  it  is  more  economical  than  the  strict 
application  of  the  definition  of  guery  evaluations. 

*  In  proceedings  ACM  SIGMOD  1979. 
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Null  Values  in  Data  Base  Management 
1.  Introduction 

Quite  frequently,  some  special  values,  referred  to  as  ‘null', 
appear  in  real  data  bases.  There  are  many  reasons  for  a  value  to 
he  null.  The  ANSI/SPAKC  interim  report  [1]  cites  14  possible 
manifestations  of  null  (e.g,  not  valid  for  this  individual,  not 
permitted  to  be  stored,  undergoing  change,  etc.).  However, 
formal  treatment  of  null  values  usually  includes  no  more  than  two 
different  interpretations.  For  instance,  in  Information  Algebra 
[2],  there  are  two  null  values,  omega  meaning  undefined,  not 
relevant,  and  theta  meaning  missing,  relevant  but  not  known. 
These  two  interpretations  seem  to  capture  reasonably  well  all 
ether  manifestations  of  null,  the  latter  usually  being  special 
cases  of  the  former. 

Very  few  data  base  systems  provide  any  support  for  null 
values.  The  data  base  designer  is  usually  very  careful  to 
prevent  their  appearance  with  specialized  schema  construction  and 
strict  rules  governing  the  insertion,  deletion  and  modification 
of  data  values.  It  is  argued  though,  that  twe  major  research 
trends,  namely,  database  translation  and  database  restructuring, 
show  clearly  the  inappropriateness  of  this  approach,  and  provide 
strong  motivation  for  a  more  formal  treatment  of  null  values.  We 
give  two  simple  examples. 

The  appearance  of  explicit  null  values  in  a  data  base  is 
sometimes  suppressed  and  made  implicit  by  the  data  base 
structure.  For  example,  consider  a  DBTG  database  with  the 


following  structure: 
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1 

1 

1 

1 

PEP  SON  (name,  age) 

I 

1 

owns 

I 

V 

1 

1 

1 

1 

SHARES  (id,  value) 

where  the  DBTG  set  'owns'  is  declared  as  optiofral-manual.  In  an 
instantiation  we  will  have  no  links  if  a  person  does  not  own 
shares  or  some  shares  are  outstanding.  The  absence  of  a  link 
indicates  an  implicit  null  value.  With  the  current  research 
interest  in  database  and  query  translations,  these  implicit  null 
values  may  become  explicit;  the  target  database  structure  being 
unable  to  hide  them.  For  example,  a  straightforward  way  to 
translate  this  network  database  to  a  relational  one  is  to  use 
two  relations: 

PERSON  (name,  age) 

SHAPES  (id,  owner,  value) 

Assuming  that  some  of  the  shares  are  outstanding,  we  have  null 
values  appearing  in  the  owner  domain.  Of  course,  there  are  other 
ways  to  translate  this  network  structure,  e.g.  with  three 
relations:  PERSON ,  SHARES,  OWNS  or  with  PERSON,  CUTSTANDING- 
SHARES,  OWNED-SHAFES ,  but  ether  problems  are  created  when  such 
approaches  are  chosen,  especially  in  modifications  [3].  For 
instance,  a  change  of  ownership  would  require  an  update  of  the 
nen-key  owner  attribute  value  in  cur  proposed  organization  - 
which  is  conceptually  expected.  On  the  other  hand,  having  three 
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relations  PERSON,  SHARES  and  OWNS,  we  may  have  to  delete  a  tuple 
from  OWNS  and  insert  another  tuple  (since  the  attribute  would  be 
part  of  the  key).  Relations  that  have  explicit  null  values  can 
also  be  used  in  resolving  the  multiple  view  support  problem  ([4], 
[6])- 

In  database  restructuring,  it  is  often  the  case,  that  a  new 
attribute  for  an  entity,  considered  previously  as  non-import  ant, 
becomes  necessary.  Its  addition  may  create  problems  if  not  all 
the  values  are  kncwn  cr  if  in  some  cases  it  is  inapplicable  to 
have  a  value  fcr  this  attribute.  For  instance,  consider  the 
relation : 

EMPLOYEE  (name,  dept,  age,  mar-status) 
and  assume  that  the  "spouse's  name"  is  to  be  added  as  a  new 
attribute  in  the  EMPLOYEE  relation.  Clearly,  for  non-married 
employees,  a  value  for  the  spouse's  name  is  not  relevant;  it 
creates  a  logical  inconsistency  in  the  data  base. 

It  may  be  that  the  introduction  of  the  "spouse's  name" 

attribute  in  the  relation  is  a  bad  design.  Whether  this  is  the 
case  or  not  should  not  affect  our  ability  to  handle  the 
representation  since  it  may  not  always  be  clear  when  we  have  a 
tad  design.  There  are  some  kinds  of  dependencies  between 
attributes  which  can  not  always  be  represented  formally.  It  may 
also  be  advantageous  from  an  economical  point  to  bo  able  to 
treat  schema  modifications  in  a  uniform  way  (e.g.  always  add  new 
attributes  to  a  relation).  An  analogy  with  programming  languages 
is  the  use  of  the  "GO  TO"  statement.  If  the  programmer  can  live 


Null  Values  in  Data  Base  Management 


127 


without  its  use,  then  that  is  fine.  But  it  should  not  be  banned 
altogether  from  the  language. 


In  this  document, 
treatment  of  null  values 
bases.  The  approach  is 
we  choose  to  use  this  mod 
because  there  is  some 
null  values  in  relational 
compared  to  this  work. 


we  will  present 
that  concentrates 
not  limited  to  the 
el  because  of  its 
work,  reported  in 
data  bases.  Thus 


an  approach  to 
on  relational 
relational  model, 
simplicity  and 
the  literature  [5] 
,  our  approach  may 


the 

data 

but 

also 

for 

be 


In  addition  to  [5],  there  are  some  other  approaches, 
appearing  in  the  field  cf  Artificial  Intelligence.  Levesque 
[10],  uses  possible  world  semantics,  the  semantics  cf  modal 
lcgic,  to  solve  the  problems  encountered  with  the  incompleteness 
cf  knowledge  in  Knowledge- Eased  Systems.  The  approach  seems 
intriguing,  but  it  is  our  contention  that,  because  of  its 

complexity,  it  goes  beyong  the  world  of  data  base  management  -  at 
least  for  the  present.  The  general  assumption  is:  given  an 

attribute  which  takes  values  from  a  certain  domain,  we  may  have 
many  degrees  of  knowledge  about  its  value.  On  the  other  hand,  in 
a  data  base,  the  situation  is  mere  simple.  Our  knowledge  about 
the  value  cf  an  attribute  is  usually  very  close  to  the  two 

extremes  (all  or  nothing).  Some  more  sophisticated  data  base 

systems  restrict  the  domain  of  values  by  imposing  certain 
constraints,  but  even  in  this  case,  the  ability  of  inferring 


knowledge  about  values  is  still  limited. 


128 


Kull  Values  in  Eata  Base  Management 

Another  approach,  similar  to  the  above,  was  introduced  by 
Lipski  [15]-  The  possible  worlds  are  called  extensions  of 
information  systems  in  this  work.  Lipski  gives  interpretations 
cf  a  guery  in  two  levels  -  the  user  level  and  the  internal  level, 
where  topological  boolean  algebras  and  modal  logic  are  used. 

2.  Ma ny-V alued  Logic  Approach  to  the  Treatment  cf  Null  Va lues 

The  next  step  from  the  very  familiar  two-valued  logic  is  the 
introduction  cf  a  third  truth  value,  a  value  which  is  neither 
true  nor  false  (e.g.  unknown).  Lukasiewitz,  Bcchvar,  Kleene  and 
the  Standard  Seguence  are  the  most  important  formulations  of  such 
logic  [7].  •  All,  except  Bochvar's,  have  identical  truth  table 
definitions  for  conjuction,  disjunction  and  negation,  differing 
only  in  implication.  There  are  two  important  facts  to  ncte  (for 
every  three-valued  logicl  • 

1.  The  three-valued  truth  tables  agree  with  the  usual  twc-valued 
ones,  when  only  true  and  false  are  involved. 

2.  Any  three-valued  tautology  (contradiction)  is  a  tautology 
(contradiction)  in  two  valued  logic,  but  not  vice-versa.  For 
example,  pv-«p  is  not  a  tautology  in  three-valued  logic. 

We  characterize  a  logic  system  as  truth-functional  if  the 
following  principle  is  always  satisfied:  For  any  formula 

F  (pi , p2 ,. . . ,pn) ,  expressed  in  terms  of  its  individual  terms,  and 
interpretation  V  (e.g.  truth- val ues) : 
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V  (F  (p1,p2,... ,pn) )  —  F  (V  (pi)  fV(p2)  . ,V  (pn))  (1) 

lor  example,  V  (p&g)  =V  (p)  Z  V  (q)  . 

Also,  V  (p v-«p)  =  V  (p)  v-*V  (p)  . 

Informally,  the  truth-functionality  principle  says  that  a 
statement  composed  of  several  substateraents  has  the  same 
interpretation  as  the  logical  composition  of  the  interpretations 
cf  all  its  substatements. 

In  order  to  talk  about  tautologies  and  contradictions  in 
many-valued  logics  we  must  designate  some  truth  values  (values 
that  can  te  accepted).  For  three-valued  logic  true  is  of  course 
a  designated  value.  If  we  want  to  be  consistent  with  the  truth- 
functionality  principle  we  can  not  designate  unknown.  On  the 
ether  hand,  if  truth-functionality  is  not  considered  essential, 
we  may  have  the  truth  value  unknown  as  both  designated  and 
antidesignated.  A  tautology  in  such  a  non-truth  functional 
system  would  be  defined  as  a  formula  that  uniformly  takes  the 
value  true  (at  least  once)  or  the  value  unknown  for  each  possible 
truth-value  assignment. 

A  treatment  cf  null  values  has  been  given  for  the  relational 
model  [5].  Here,  only  one  null  value  is  considered,  '2'  meaning 
•missing'  or  'value  presently  unknown*.  Codd  adopts  a  three¬ 
valued  logic  for  use  in  extracting  data  from  data  bases  with  null 
values.  Ihe  same  symbol  *2*  is  used  to  denote  the  unknown  truth 
value,  and  in  fact  'S'  corresponds  to  theta  of  Information 
Algebra.  The  whole  approach  is  based  upon  a  'null  substitution 
principle'  that  gives  conditions  under  which  a  truth-valued 
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expression  takes  the  value  a).  The  claim  is  that  the  truth- 
functional  three-valued  logic  is  consistent  with  this  principle. 
Ccdd  further  applies  the  principle  to  egualit y/inegua lity 
testing,  set  membership,  set  inclusion,  etc.  The  relational 
operators  are  also  reexamined  for  the  possibility  of  null  values 
in  relations.  Generally,  a  relational  guery  (either  expressed  in 
the  relational  algebra  cr  with  predicate  calculus  formulas)  has 
two  results..  A  true-resu It  (all  tuples  for  which  the  specified 
guery  condition  evaluates  to  T)  and  a  maybe  result  (the  condition 
evaluates  to  S) . 


A  data  model,  which  is  by  itself  incomplete,  is  expected  to, 

at  least,  be  precise  for  what  it  car  describe.  It  has  to  give 

non  misleading  replies  when  gueried;  no  more  nor  less  than  it 

knows.  There  are  some  cases  where  the  null  substitution 

principle  is  not  sufficient  as  the  model’s  tool  for  a  precise 

description  of  the  situation,  as  was  first  observed  in  [11].  The 

use  of  this  principle  implies  that  familiar  identities 

(tautologies,  theorems  or  provable  formulas)  will  fail  to  hold, 

since  we  only  designate  the  value  true  in  this  three-valued  logic 

fcrmulaticn  (for  the  true -result  of  a  guery).  For  instance, 

consider  the  relation: 

EPELCYEE (name, dept , age, mar_status) 

and  the  relational  calculus-like  guery: 

LIST  EMPLOYEE. NAME  WHEFE 

(EMPICYEE. AGE>50)  OR  ( EM  PLOY  EE. A GE<5 0) 

fchen  tjie  employee's  age  is  unknown,  the  name  returned  by  the 

guery  will  be  in  the  may be-result.  But  the  guali f icat ion 

condition  is  a  tautology  (all  possible  ages  are  considered)  and 
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the  guery  is  expected  to  list  all  employee  names  (t rug- resu  It )  . 
Consider  new  another  guery: 

"Feturn  all  married  employees  that  work  in  the  research 
department,  and,  all  single  employees  over  forty”. 

Expressed  in  a  relational  calculus- like  language: 

I 1ST  EMPLOYEE  WHERE 

(  (EMPLOYEE .DEPT=RES EARCH)  AND  (EMPLOYEE. MAR-STATUS=M)  ) 

OF  ( (EMPLOYEE.  AGE  >  40)  AND  (EMPLOYEE..MAR-STATOS=S)  ) 

Its  gualif icat ion  condition  can  be  written  as:  (pEq)v  (r6-»q) 

where,  p , g , r  are  the  simple  predicates: 

(EMPLOYEE. DEPT=RES EAR CH)  ,  (EMPLOYEE..MAR-STATOS=M)  ,  and 

(EMPLOYEE.  AGE>40)  respectively.  If  p  and  r  have  equal  truth- 

values,  then  the  value  of  the  formula  is  independent  of  the  value 

cf  q  and  is  egual  to  the  value  cf  p  (cr  r) .  This  fact  is 

expressed  by  the  theorem:  (p=  r)  =>  (  (p5q)  v  (r&-*g )  =p)  ,  where  '=>' 

stands  for  ’implies'.  If  g  is  unknown  the  theorem  does  not  hold 

in  the  three-valued  logic  used  by  Codd.  Hence,  the  guery  will 

evaluate  to  unknown  for  the  tuple  (Martin,  research,  50,  2).  But 

Martin  is  either  married  or  single.  If  he  is  married  the  first 

term  cf  the  OR  condition  evaluates  to  true  and  if  he  is  single, 

the  second.  Therefore,  the  tuple  should  be  in  the  tr ue-resul t  of 

the  query.  Notice  that  in  our  intuitive  considerations,  the 

principle  (1)  does  not  hold.  That  is: 

V  (F) *F  (V ( pi)  , V  (p2) , _ _ V  (pn) ) .  For  instance,  with  F  :=  pv-p  we 

have  V  ( F )  =  tr  u  e  and  F  (V  (p))=V  (p)v-’V  (p)=un known  when  V  (p)  =  unknown. 

In  this  respect,  we  have  a  non  truth- functional  approach. 

We  mentioned  in  the  introduction  that  there  are  other 
manifestations  of  null  values  in  addition  to  'missing'.  If  we 
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attempt  a  generalization  of  the  three-valued  logic  used  by  Codd 
tc  four  or  more  valued  logic,  we  run  into  many  problems.  All 
approaches  to  many-valued  logic  have  been  mainly  abstract.  We 
are  dealing  with  assignments  of  values  that  are  unspecified. 
They  may  not  even  be  truth-values  at  all.  Notice  here  the 
difference  from  the  two-valued  logic,  where  the  truth  assignments 
are  preexisting  criteria,  to  which  the  axiom  system  must  conform. 
Working  with  'abstract*  logic  is  quite  interesting  theoretically, 
but  the  problem  of  a  semantic  interpretation  is  considered  by 
logicians  generally  unsolved.  We  may  be  able  tc  axiomatize  a 
four-valued  logic,  but  we  face  problems  with  attempts  to  give  one 
particular  semantic  interpretation  to  the  truth  values. 

3.  The  use  of  Denotation a  1  Semantics  for  the  trea tmen t  of  mill 

The  Eenotational  Semantics  theory,  developed  by  Scott  ([8], 
[12]),  provides  a  formal,  "mathematical"  semantics  to  programming 
languages.  The  motivation  for  using  denctational  semantics  in 
cur  treatment  of  null  values  will  be  outlined  in  our  brief 
discussion  of  the  theory. 

There  are  two  basic  results  of  denotational  semantics.  The 
first  result  states  that  a  data  type  is  a  complete  lattice,  and 
the  second  states  that  "allowable"  mappings  between  data  types 
are  continuous  (preserve  limits).  The  concept  of  approximation 
is  central  in  the  theory.  Its  use  is  motivated  by  the  fact  that 
data  types,  traditionally  considered  as  sets  of  objects, 
naturally  include  infinite  objects  for  which  no  finite 
representation  can  be  given.  We  can  define  these  infinite 
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cljects  as  the  limits  of  a  sequence  of  finite  approximations. 
Very  informally,  an  element  of  a  data  type  is  said  to  approximate 
another  element,  if  this  second  element  is  mere  'accurate*  than 
tie  first  one.  For  instance,  consider  the  very  simple  example  of 
intervals  (that  could  fce  used  to  define  real  numbers).  We  say 
that:  x  ag:  y  or  [x1,x2]  a j:  [y1,y2]  iff  x1<y1<y2<x2  (i.e.  one 
interval  is  inside  the  other)..  The  way  approximation  is  defined, 
it  is  trivially  shewn  that  it  is  reflexive,  transitive  and  anti¬ 
symmetric.  Therefore  a  data  type  together  with  this 
approximation  relation  is  a  partially  ordered  set.  The  infinite 
objects  (limits)  are  naturally  the  lowest  upper  bound  (lub)  and 
greatest  lower  bound  (gib)  of  such  sets.  Returning  to  our 
example,  we  have  the  infinite  interval  which  approximates  every 
ether  interval  as  gib,  and  the  empty  interval  which  is 
approximated  by  any  other  interval  as  lub.  These  elements  are 
usually  referred  to  as  bot  or  undetermined  or  -*•,  and  top  or 
ever deter  mined  or  T.  In  summary,  a  primitive  domain  or  data  type 
is  defined  in  the  theory  as  the  lattice: 

<  D  IT  {  top,  bot  },a£>  or  <  D°  ,  a_p  > 

Considering  approximation  as  the  information  content  of  the 
domain  elements  (how  accurate  they  are),  then  it  is  obviously  not 
desirable  to  allow  mappings  between  domains  that  increase  the 
information  content  of  an  element.  Hence,  we  restrict  mappings 
between  domains  to  be  continuous.  In  other  words,  preserve  the 
approximation  relation  and  the  limits  o±  the  domains.  The  notion 
of  continuous  functions  comes  in  handy  with  the  notion  of 
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effectively  computable  functions,  functions  that  produce  a  result 
after  considering  only  a  finite  amount  of  input. 

Modelling  the  real  world  (e.g.  with  a  data  tase)  is,  of 
course,  always  an  incomplete  operation  [10].  The  reason  for  this 
incompleteness  is  that  we  need  infinite  information  in  our  data 
base.  Instead,  we  can  only  approximate  this  infinite  information 
sc  that  we  may,  after  a  process  of  finite  approximations 
(gathering  more  and  more  data),  end  up  with  a  satisfactory  model 
of  the  real  world.  Consider  a  data  base  with  initially  no 
values  in  it.  At  this  stage,  our  model  describes  an  infinite  set 
cf  worlds,  since  all  the  nonexistent  values  or  null  values  in  the 
data  tase  may  approximate  or  describe  any  situation.  It  can  be 
said  that  it  contains  no  information,  except  the  fact  that  some 
attributes  refer  to  objects  (according  to  the  schema) .  This 
state  is  represented  as  unknown.  Inserting  more  and  more  values 
into  the  data  base  we  get  a  clearer  picture  of  the  world  we  are 
modelling.  In  other  words,  we  get  a  better  approximation  of  the 
world.  We  can  only  have  finite  (possibly  good)  approximations 
fcr  the  real  world.  The  infinite  situation  can  only  be 
represented  in  our  data  base  as  an  inconsistent  object  (we  have 
nc  tools  tc  represent  it  in  a  finite  way)  . 

We  now  concentrate  on  the  domain  of  truth  values  in  the  well 
known  two-valued  logic.  The  domain  is  T  =  {true,  false]  and  as 
a  data  type  can  be  extended  in  denotational  semantics.  Consider 
the  values  true  and  false  as  alternative  answers  of  a  guery  and 
the  approximation  relation  defined  in  T.  An  element  in  T  is 
approximated  by  another  element  in  T  iff  it  is  a  more  accurate 
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answer.  Trivially,  true  and  false  are  not  related  with  the 
approximation  relation.  Incorporating  the  lub  and  gib  in  T  we 
have: 

T o  =  T  U  {bot ,  top} 

Since,  by  definition,  hot  approximates  every  element  in  T ,  it 
is  implied  that  it  could  either  be  true  or  false.  However,  there 
is  currently  not  enough  information  to  decide  which  of  the  two. 
Therefore,  a  reasonable  semantic  interpretation  for  bot  is 
unknown.  Cn  the  other  hand,  since  top  is  approximated  by  every 
element  in  T,  it  is  implied  that  it  is  both  true  and  false. 
Semantically,  this  would  mean  that  we  have  an  inconsistent 
element. 


Like  the  domain  of  truth  values,  all  the  domains  (data  types) 
we  will  consider  in  our  application  of  the  theory  are  going  to  be 
flat.  That  is,  all  elements  in  the  domain  (except  top  and  bot) 
are  not  related  with  the  approximation  relation.  The  reason  we 
are  only  interested  in  flat  lattices  will  become  apparent  in  our 
discussion  of  the  relational  model  extension. 


We  new  give  rules  for  function  extensions  between  domains, 
such  that  these  extensions  are  continuous  functions.  Then,  as  an 
illustration,  we  show  how  some  well  known  functions  are 
extended. 


let  E 1 , D2  ,  .  . .,Dn 
denctational  semantics 
incorporation  of  the 
and  top  elements  may 


and  D  be  domains  and 

be:  D  1° , D 2°  ,Dn° 

bot  and  top  elements) 
be  different  in  each 


their  extensions  in 
and  D°  (with  the 
.  Note  that  the  bot 
domain,  but 


9 


for 
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simplicity,  we  will  assume  that  we  always  have  the  same  elements 
denoted  with  -*■  and  T  respectively,.  Suppose  then,  that: 

f  :  E1xE2x...xEn  - >  D  is  a  (partial)  function. 

We  extend  f  (denoted  by  f°,  a  total  function)  as  [9]: 

f°  :  E  1°xD2°x. . . xDn° - >  D°  :  f 0  (dl  ,d2, . . . , dn)  :=  d 

according  to  the  following  rules: 


1.-  if  Vi,  1<i<n,  we  have  di€Di,  then: 

(a)  if  f  is  defined  on  dl  ,d2  .. ,dn,  then 

f  0  (dl  ,d2,.  .  .  ,  dn)  :=  f  (d  1 ,  d2 dn) 
as  expected,  the  extension  is  the  same  as  the  original 
function  in  the  original  domains. 


(b)  if  f  is  undefined  on  d 1 , d2  . . , dn,  then 

f 0 (dl ,d2, - , dn)  :=  T 

2.-  if  di  =  T  for  an  i,  1<i<n  then: 

£°  (d1,d2,..  . ,  «  ,dn)  :  — 

2.-  if  di  =  J-  and  Vj,  j#i,  dj€Dj  then: 


T 

if  f  (dl ,  d2, . . .  ,e,  .  . .  ,dn) 

is  undefined 
Ve,  e€Ei 


f  o  (dl  ,d2. 


dn ) 


♦-  c  ,  c  in  D 
1  if  f  (dl ,  d2  . .  ,e,.  .  . ,  dn) 

J  is  either  undefined  or 

j  equals  c  (at  least  once) 

|  Ve,  e6Di 


+-  x 

in  any  other  case 


We  consider  f  to  be  in  general  a  partial  function  (unless 
cthervise  stated).  Its  extension,  f°,  will  always  be  a  total 


in  rj 
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function.  It  is  easy  to  verify  that  the  function  f°  as  defined 
above  is  continuous,  see  [1*1]  for  a  proof. 

As  an  example,  we  define  the  extended  fgreater  than' 
function.  Let  Dl,  D2  be  two  domains  (compatible).  ’greater'  is 
defined  as: 

greater: D 1 xD2 - >  1  :  (a,b) - >  greater  (a, fc)  :=  a>b 

Its  extension  is: 

greater0: Dl °xD2° - >  T°  :  (a,b) - >  greater 0 (a, b)  : 

1.  greater0  (a, b)  :=  greater  (a, b)  if  a€Dl,  b€D2 

greater 0  (a ,b)  :=  inconsistent  if  either  a  or  b  is  x 
let  a  te  -* 


-  true  if  ¥e  in  Dl 


9 


b  in  E2 
=  true 


greater (e,b) 


greater0  b)  :  = 


|-  false  if  ¥e  in  Dl,  b  in  D2 
|  greater. (e, k)  -  false 


-  unknown  in  any  ether  case 
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Vie  summarize  the  definitions  of  (the  extended)  conjunction, 
disjuicticn  and  negation  with  the  following  tables: 


ANDO 

1 

true 

fa  lse 

unknown 

inconsist 

true 

1 

true 

false 

unknown 

incorsist 

false 

1 

false 

fa  lse 

f  al  se 

in consist 

unkno  wn 

1 

unknown 

fa  lse 

unknown 

inconsist 

inconsist 

I 

inccnsist 

inconsist 

inconsist 

incon^ist 

CTO 

\ 

true 

false 

unknown 

incorsist 

true 

I 

true 

true 

true 

incorsist 

false 

1 

true 

fa  lse 

unknown 

inconsist 

unknown 

1 

true 

unknown 

unknown 

inconsist 

inconsist 

1 

inconsist 

inconsist 

inconsist 

inconsist 

NCT° 

1 

true 

false 

unknown 

inconsist 

1 

false 

true 

unknown 

incorsist 

Ccdd» 

con  si 

canno 

v«ith 

piinc 

guery 

letwe 


t  may  be  observed  here  that  these  tables  are  identical  to 
s  truth-tables  when  only  true,  false  and  unknown  are 
dered.  The  important  thing  we  stress  though,  is  that  we 
t  use  these  tables  for  the  evaluation  cf  any  formula  (query) 
the  usual  evaluation  algorithms  and  the  null  substitution 
iple,  since  we  have  a  non-truth-functional  system.  The 
evaluation  will  later  be  defined  as  a  continuous  function 
en  data  types. 
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4.  The  Relational  Mode  1  Revisited 

We  now  concentrate  on  the  relational  model  and  we  discuss  how 
the  inclusion  of  the  bot  and  top  elements  affect  our  notions  of 
relation,  guery,  etc.  Traditionally,  relations  correspond  to 
predicates  in  Rredicate  Calculus.  A  tuple  is  either  in  a 
relation  (R(t)=true)  or  it  is  not  (R  (t)  =f  alse) .  We  incorporate 
the  top  and  bot  elements  (values)  in  each  domain..  Hereinafter, 
we  will  use  a)  to  denote  bot  and  £  to  denote  top  when  we  are  using 
them  as  values  in  relations.  A  possibly  acceptable  semantic 
interpretation  of  these  top  and  tot  elements  in  attribute  values 
of  a  relation  is  nothing  and  missing  respectively.  The  a  (null) 
value  can  be  thought  of  as  a  placeholder,  and  the  <£  (null)  value 
can  be  thought  of  as  non-applicable.  Hence,  a  relation  is  now 
defined  as  follows: 

R0:D1°xD20x...xDn°  - >  T° ,  Di°  =  Di  0  {3,*},  1<i<n 

ft n  important  fact  we  stress  is:  in  all  domains  that  we  use 
for  relations,  all  elements  (with  the  exception  of  2  and  2)  are 
equally  accurate  (have  the  same  information  content).  That  is, 
they  are  not  related  with  the  approximation  relation  (flat 
lattices).  Stated  in  ether  words,  one  of  the  following  holds  in 
a  domain  of  attribute  values: 

1.  We  know  the  (correct)  value  cf  an  attribute; 

2.  we  have  no  knowledge  about  the  value  of  an  attribute  (3)); 

2.  too  much  is  known  about  the  value  of  an  attribute,  making  it 


inconsistent  (tf)  . 
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Fcr  the  reader  that  may  have  been  confused  with  our  notation, 
here  is  a  summary  of  the  symbols  we  use: 


denctaticral  | semantic  | symbols  used  Jsemantic 

semantic  symbols  |meanings  in  | in  a  relation  1  meanings  in  a 


Idomain  T 

Idomain 

Irelation 

-l 

lot 

unknown 

2 

missinq 

T 

top) 

inconsistent 

0 

nothin  q 

Vie  now  give  a  concrete  example  of  an  extended  relation: 
EMPLOYEE  (name,  dept,  age,  mar-status,  spouse  * s-na me) 
Consider  an  instance: 


name 

1 

dept 

1 

age  | 

mar-status 

1 

spouse  *s-name 

1 

Eenni s 

1 

Advert. 

1 

2 

1 

1  M 

1 

V  icky 

1 

M  arc 

S 

3 

1 

2 

S 

M 

I 

Brenda 

1 

Ered 

1 

Besearch 

S 

29 

I 

S 

1 

0 

1 

Martin 

1 

Eesearch 

1 

55 

1 

2 

! 

2 

1 

Dennis*  and 

Marc  *  s 

ages 

are  missinq.  In 

addit 

ion,  we 

presently  do  not 

know  the 

department  in  which  Marc 

works. 

Notice 

that  Ered  is 

single 

and 

asking  for  his  wife's 

ra  me  is 

inapplicable.  A 

value 

at 

this  point  would 

ere 

ate  an 

inconsistency.  Therefore 

,  we 

have  nothing.  The 

usua  1 

rule  for 

relations  (no  two 

tuples 

are 

identical)  holds  in 

an 

extended 

relation,  with  2  and  0  considered  as  ordinary  values. 

A  query,  which  retrieves  relations,  corresponds  to  predicate 
calculus  formulas  with  free  variables.  Vie  assume  that  Q  is  a 
total  function.  Using  denotational  semantics  and  our  basic 
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framework  fcr  function  extensions,  we  define  the  guery  evaluation 
as  a  function: 

C°  :  E 1 0  x E2°  x. . . xDn°  - >  T° 

ar  extension  of  Q  :  D1xD2x...xDn  - >  T 

1-  Q°  (d1,...,dn)  :=  Q(d1,...,dn)  if  -Vi,  1<i<n,  di€Di 

The  result  of  the  extended  query  is  identical  to  the  result 
cf  the  conventional  guery,  when  nc  null  values  are  present. 


2.  Q°(d1,..«,dn)  inconsistent  if  ^i  :  di  =  ft 


2.  Let  di  =  S 


true  if  g  (di, _ ,e, ..  .,dn) 

equals  t£ue  Ve,  eGDi 

C  0  (d  1 , . . . ,  di  , . . . ,  dn)  :=  |  false  if  Q  (di , . . . , e, . . . , dn) 

equals  false  Ve,  e€Di 

+  unknown  otherwise 

Our  evaluation  function  is  continuous.  Notice  the  treatment 
of  the  ft  (nothing)  null  value.  According  tc  our  defiriticn,  any 
occurrence  of  ft  as  an  attribute  value  in  a  tuple  (where  the 
attribute  is  referenced  in  the  condition  of  the  query),  will 
trivially  result  in  rejection  of  the  tuple  as  a  candidate  for  the 
true  cr  maybe  result  of  the  query.  These  are  the  only  results 
that  the  user  sees.  Conversely,  no  question  can  be  asked  of  the 
data  base  involving  the  nothing  value.  For  instance,  the  query: 
"return  all  employees  for  which  the  spouse’s  name  is  nothing "  is 
invalid.  It  is  our  contention  that  gueries  have  meaning  only 
when  they  refer  tc  something  (possibly  missing  but  still 


something  that  can  take  a  value).  Summarizing,  nothing  does  not 
exist  as  a  value,  as  far  as  the  user  is  concerned. 
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We  no*  give  two  example  queries  based  on  the  instantiation  of 
the  previously  listed  relation  EMPLOYEE.  The  relation  has  four 
tuples  which  we  denote  with:  t1f  t2,  tl,  t4,  respectively. 

3 )  LIST  EMPLOYEE  WHEEE 

(EMPLOYEE. AGE  >  50)  CE  (EM  PLCYEE.  -AGE  <  50) 

gc;Do - >  TO  :  (AGE)  0  >  TO  :  a - >  QO(a)  :  = 

:=  OE°  (greater0  (a,  50) ,  NOT0  (greater 0 (a,  50))) 

where  D  is  the  domain  AGE,  a  is  a  value  in  (AGE)0 

C°(a)  :=  true  for  all  the  tuples.  Clearly,  it  evaluates  to  true 
fcr  t3,  t4  .  Consider  the  first  tuple  (tl).  For  every 
conceivable  age  in  the  domain  of  employee  ages  (e.g.  15<age<65), 

the  query  C  would  evaluate  to  true.  Similarly,  Q°  evaluates  to 
true  for  the  second  tuple. 

t)  Consider  the  guery: 

IIST  EMPLOYEE  WHEEE 

(  (EMPLOYEE.  DEPT=EESEAECH)  AND  (EMPLOYEE .  MAB-',STATUS=M)  ) 

CE  ( (EMPLOYEE. AGE  >  40)  AND  (EMPLOYEE. MAE- ST AT 0 S  =  S) ) 

Its  evaluation  is  defined  as  a  continuous  function: 


Cc:Dl0xD20xD30 - >T°  :  (DEPT) 0 x (AGE)  °x  (MAE-STATOS)  0 - >T°  : 

:  (d,a,s)  — — >  C°(d,a,s)  :=  OE°  (AND0  (egual0  (d,  EESEAECH), 

equal°(s,  M) ) ,  AND0  (  greater0  (a,  40),  egual°(s,  S))) 


where  d€D EPT° ,  a6AGE° ,  s€ MAE-STATOS 


Let  us  determine  whether  t4  =  (Martin,  research,  55,  5),  a)) 
belongs  to  the  true  result  of  the  guery.  The  values  for  d,  a  are 
kcwn.  The  only  missing  value  is  for  the  marital  status.  The 
domain  MAE-STATOS  =  {M,  S) .  According  to  our  definition. 


{M,  S). 
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C°  (research,  55, 5))  is  true  if  Q  (research, 5  5 ,  e)  is  true  for  every  e 
ir  MAE- STATUS. 

i)  e=K 

C  (research ,  55 ,  M)  =  OF.  (AND  (true  ,  true)  ,  AND  (true  /false) ) 

=  OR  (true,  false)  -  true 

ii)  e=S 

C  (researc  t ,  55,  S)  =  OP.  (AN  E  (true,  false)  ,  A  ND  (true,  true) ) 

=  OR  (false,  true)  =  true 

Hence  t4  is  in  the  true-result  of  the  query. 

The  reader  can  easily  verify  that: 

C°=  false  for  1 1 , 1 3  and  0 °=  wn  for  t2. 

We  now  discuss  the  differences  of  this  approach  with  the  one 
presented  in  [5].  A  tuple  will  be  in  the  true-result  (denote 
TP  (Q) ,  a  set)  of  a  query,  if  and  only  if  we  can,  using  the 
information  available  from  the  data  base,  determine  that  the 
condition  of  the  query  evaluates  tc  true  for  the  particular 
tuple.  Similarly,  a  tuple  is  in  the  maybe-result  (denote  MR  (Q ) ) 
if  and  only  if  the  query  condition  evaluates  to  unknown  for  the 
tuple.  We  have  shown  that  ir.  general: 

TP*  (Q)  subset  of  TP  (Q )  and  MP(Q)  subset  of  ME*  (Q) 
where  the  *  indicates  Codd’s  results. 

This  fact  makes  the  approach  with  the  use  of  Eenotaticnal 
Semantics  more  precise.  We  get  mere  information  from  the  data 
base.  locking  at  it  from  another  angle,  it  is  shown  that  the 
rules  Codd  uses  for  the  evaluation  of  a  guery  are  "sound"  but 
net  "complete".  If,  using  the  null  substitution  principle,  we 
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can  de 
is  n  c 
"  n  o  M  a 
is  nc 
con  fid 
the  m  a 


termine  that  a  tuple  belongs  to  the 
dcubt  that  it  really  belongs  ther 
newer,  i.e.  'the  tuple  does  not  bel 
t  always  correct  -  we  dc  not 
ence  in  a  "nc"  answer.  Similar  res 
y be-result . 


true-result,  then  there 
e.  Cn  the  other  hand,  a 
ong  to  the  true-result', 
have  the  same  level  of 
ults  can  be  drawn  for 


In  addition,  we  have  introduced  the  i neons is tent -res u It , 
which  is  trivially  evaluated.  If  a  tuple  has  at  least  one  2 
value  for  an  attribute  and  this  attribute  is  referenced  in  the 
guery  condition,  then  the  tuple  belongs  tc  the  inconsistent 
result. 


Until  now,  in  our  examples,  we  evaluated  gueries  using  their 
definitions  as  continuous  functions.  It  is  of  course  a  very 
inefficient  method.  If  we  have  a  tuple  with  m  null  values,  we 
need  ir  general  at  most  N 1xN2x.~ . xN m  substitutions  for  the  guery 


evaluation 

(where 

Ni 

is  the  numbe 

r  of  val 

ues  in  the  domain 

Di)  . 

In  the  remaining 

part 

of  this 

section 

we  will  develop 

an 

a  Igor ithm 

for  query 

evaluation 

which  is 

mere  economical. 

The 

algorithm 

is  based 

on 

transforma ti 

ons  of 

gueries  to  suit 

able 

forms  (symbolic  manipulations).  For  simplicity,  we  will  drop 
from  cur  notation  the  '°*,  but  the  reader  should  always  bear  in 
mind  that  we  are  talking  about  complete  lattices  and  continuous 
functions. 


The  atomic 
following  form: 

pi  :  = 


formulas  or  primitive  terms  in  a  guery  have  the 


(attribute-name  o_p 


v) 
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where  cj:  is  ere  cf  the  predicate  symbols  =  ,  * ,  <,  <,  >,  >  and  v 
is  a  value.  Clearly,  a  primitive  term  is  a  continuous  function 
and  the  same  rules  for  its  evaluation  hold.  Next,  we  present  a 
series  of  theorems  (proofs  are  straightforward  using  induction  on 
the  lenqth  of  the  query,  see  [14]).  For  all  gueries  we  will 
assume  that  the  only  partitions  of  an  attribute  domain  are  given 
by  p  and  -*p. 


Ihegrem  1 

let  C  be  a  query  with  the  special  form: 

Q  : =  pi vp2v. . . vpn 
where,  for  each  pi: 

(a)  pi  is  a  primitive  term 

(b)  -«pi  does  not  belong  in  C 

Then  for  each  tuple  t=  (d  1  ,32,...  .  ,  dn)  where  di  *2,  Vi,  1<i<n 
Q  (t)  =  true  iff  -§pi  :  pi(di)  =  true 

Theorem  2 

let  C  be  a  query  with  the  special  form: 

Q  :  —  p1Sp26. . .  Spn 
where,  for  each  pi: 

(a)  pi  is  a  primitive  term 

(b)  -«pi  does  not  belong  in  Q 

Then  fer  each  tuple  t=  (d 1  ,d 2, .. . , dn)  where  di*2,  ¥i,  1<i<n 
Q  (t)  =  false  iff  -jpi  :  pi(di)  =  false 

The  above  theorems  say  that  if  a  query  is  a  disjunctive 
(conjunctive)  normal  ferm  of  primitive  terms,  then  with  at  most  n 
operations  we  can  determine  whether  the  guery  has  the  value  true 
(false)  for  a  particular  tuple.  These  results  capture  well  our 
intuitive  ideas  for  the  value  cf  a  guery.  Naturally,  the  next 
step  is  to  consider  the  Principal  Disjunctive  Normal  form 
(PDNF(Q))  and  Principal  Conjunctive  Normal  Form  (PCNF(C))  of  a 
guery  Q.  let  us,  very  briefly,  recall  their  defiritiens  [13].  A 
m interm  mi  is  any  product  (series  of  conjuctions)  of  the 
primitive  terms  or  their  negations.  None  of  the  minterms  contain 
let h  a  primitive  term  and  its  negation.  All  possible  minterms 
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fcr  n  primitive  terms  are  2**n  and  there  is  an  ordering  between 
them.  Fcr  instance,  with  n=2,  we  have  the  following  minterms  (in 
order):  p1Sp2,  p1S-«p2,  -<p18p2,  -*p1£-*p2.  Having  this  order  we 
can  talk  about  the  i-th  minterm.  The  PDNF  (C)  is  a  sum  (series  of 
d is jurcticns)  cf  minterms.  Using  the  order,  we  denote: 

trHF(C)  :=  Z  i1,i2, _ ,ik  ,  0<il<  (2**n)  -  1 ,  1<k<n 

Using  duality  we  can  obtain  the  maxterms  and  the: 

TCNF(C)  :=  TT  jl, j2,. , jn  ,  0< jl<  (2**h)-1,  1<m<r 

A  recursive  algorithm  that  transforms  a  guery  Q  tc  PDNF(Q)  is 
given  in  [13].  The  PCNF  (C)  can  be  obtained  from  the  PDNF(Q) 
using  duality. 

Theorem  3 

Let  the  FDNF  (£)  and  the  tuple  t=  (d 1 , d2  , . . . ,dn)  ,  di*2  ,  1<i<n, 
then:  Q(t)  :=  FDNF(Q)  (t)  =  false  iff  -Vminterm  mj,  m j  (t)  =  false 

1  he or era  4 

let  the  PCNF  (C)  and  the  tuple  t=  (d  1 ,  d2 .  .  , dn)  ,  di#jz,  1<i<n, 
then:  Q(t)  =  PCNF  (Q)  (t)  =  true  iff  ¥maxterm  Hi,  Mi  (t)  =  true 

The  above  results  suggest  the  followirg  algorithm  for  the 
guery  evaluation: 

Algorithm  Query  evaluation 
1.  Create  the  PENF(Q),  PCNF(Q) 

^ -  for  each  t  =  (d1,d2,...,dn) 

/*  determine  which  result  it  is  in  */ 

3.  i.f  (  -]di  :  di  =  £) 

/*  the  guery  evaluates  to  inconsistent  */ 
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tjben  continue  f  r  cm  2 
4 „  if  (  V  min term  mj,  m j  (t)  =  false  ) 

/*  the  query  evaluates  to  false  */ 
then  continue  from  2 

5.  if  (  ¥  maxterm  Mj,  j  ( t)  =  true  ) 

/*  the  query  evaluates  to  true  */ 
then  TF(Q)  =  TF  (Q)  D  {t} 

/*  else  the  query  evaluates  to  unknown  */ 
else  ME(C)  =  MF(Q)  U  {t} 

6.  continue  from  2 

As  an  illustration  of  the  use  of  the  above  algorithm  we  will 
evaluate  the  query: 

"Feturn  all  employees  that  work  ir  the  Advert,  department  and  are 
cider  than  50  ,  and,  all  employees  that  are  married  and  their  age 
is  less  than  or  equal  to  fifty.” 

In  a  relational  calcu lus- like  language: 

LIST  EMPLOYEE  WHEFE 

(  (EMELOYEE. AGE>50)  AND  (EMPLO YEE. DEPT=AD VEST) 

OF  (EMPLOYEE. AGE150)  AND  (EMPLOYEE. ST ATUS=M) ) 

For  simplicity,  we  will  use: 

pi  :=  (EMPLOYEE. AGE>50) 
p2  :=  (EMPLOYEE.  DEPT=ADVEFT) 
p3  :=  (EMPLOYEE. STATUS=M) 

Fence,  our  condition  is:  Q  :=  (pi  8p2)  v  (-»p15p3 ) 

FCNF  (C)  =  "ft"  0 , 2, 4 , c  = 

(p1vp2vp3)  8  (  p  1  v-»p2  vp3 )  8 
(-*p  Ivp 2vp3 )  F  (~«p1  vp2v-»p3) 

PENF(C)  =  2.  1  ,3,6,7  = 

(pi 8p28p3) v (pi €p28-p3) v 
(— »p  18p 28p3)  v  (-p18-p28p2) 
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Notice  that  we  only  create  the  normal  forms  once  (not  for 
each  tuple).  let  us  evaluate  Q  on  the  first  tuple  tl  of  our 
example  relation.  In  our  case  d1  =  a),  d2=A  D VEB 1  ,  d3=K.  We  have  no 
values,  sc  we  proceed  in  step  4.  The  first  minterm 
(m  1=p  18p26p3)  does  not  evaluate  to  false ,  since  there  is  no  pi 
such  that:  pi  (di)  -false  (pi  (®)  =  unkncwn,  r2  ( AEVEPT)  =true, 

p2(M)=true).  Hence,  we  eliminate  the  case  of  tl  fceirg  in  the 
f a lse-res  ul t  and  we  proceed  to  step  5 .  For  every  maxterm  M j,  we 
have  at  least  one  pi  such  that:  pi(di)=true.  Therefore,  tl 
belongs  to  the  t rue-result  (TE  (Q ) ) .  In  the  same  way  we  can 
evaluate  C  on  the  other  tuples  of  our  example  relation. 

It  is  obviously  more  economical  to  use  this  method  for  query 
evaluation,  since  we  do  not  have  to  substitute  all  possible  ages 
for  each  tuple  in  the  relation.  Actually,  for  this  example,  the 
domain  (ages)  is  relatively  small  (ages=[  15,65  ])  .  The  situation 
would  have  been  more  indicative,  if  we  were  considering  very 
large  domains  (e.g.  names).  The  disadvantage  of  this  algorithm 
is  that  the  length  of  the  normal  forms  grows  exponentially  with 
the  number  of  primitive  terms  (2**n).  For  simple  queries  with  a 
small  number  of  primitive  terms,  it  would  still  be  economical.  On 
the  ether  hand,  the  presence  of  quantifiers  (replaced  by 
repetitive  disjunctions  or  con j unct ions)  would  make  the  length  of 
the  normal  forms  unmanageable. 
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:.  Concluding  Femar ks 


In  this  paper  we  presented  a  formal  treatment  cf  null  values 
in  relational  data  bases  using  denotational  semantics.  We 
started  with  a  three-valued  logic  approach  [5],  which  is  truth- 
functional,  and  we  established  cases  where  the  result  cf  its 
application  failed  tc  capture  our  intuitive  notion  of  truth. 
That  led  us  to  an  approach  where  the  truth  values  are  preexisting 
criteria.  It  came  as  no  surprise  that  the  approach  was  non 
truth-functional.  The  use  of  denctational  semantics  provided  us 
with  a  very  powerful  tool  in  mapping  abstract  notions  to  their 
•meanings'..  In  addition,  the  lattice- theoretic  approach  is  on  a 
par  with  mere  conventional  approaches  to  the  theory  of 
computation  [12].  £11  domains  (abstract  notions  and  meanings) 
are  ccmplete  lattices  and  the  mappings  (queries)  are  total 
continuous  functions.  Cur  emphasis  on  total  continuous  functions 
is  their  close  relationship  tc  computable  functions.  The 
requirement  of  dealing  with  ccmplete  lattices  gave  us  the  ability 
cf  having  two  semantic  interpretations  of  a  null  value  (or,  in 
ether  terms,  havirg  two  distinct  null  values:  missing  and 
nothing).  We  presented  an  "extended”  uel aiional  model,  where  the 
relations  have  null  values,  and  we  defined  the  evaluation  of  any 
query  as  a  continuous  function  with  range  the  domain  cf  truth 
values.  It  is  important  tc  ncte  that  other  ways  of  defining  the 
evaluation  of  a  query  exist  (ether  continuous  functions).  In 
particular,  our  treatment  of  the  null  value  nothing  may  not  be 
universally  acceptable.  On  the  other  hand,  as  long  as  the 
evaluation  function  used  is  continuous  and  properly  described. 
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the  results  of  the  evaluation  are  within  the  framework  of  the 
theory.  This  fact  justifies  our  use  of  denotational  semantics  as 
a  very  flexible  tool  for  a  formal  treatment  of  null  values. 

Finally,  realizing  the  irrpract  ical  i  ty  of  applying  the 
definition  cf  the  evaluation  function  in  a  straightforward  way, 
we  presented  an  algorithm  for  a  more  economical  query  evaluation. 
The  algorithm  is  based  on  symbolic  manipulations.  The  query  is 
first  transformed  to  its,  well-known  propositional  calculus 
forms,  PDNF  and  PCNF.  Next,  it  is  established  whether  a 
particular  tuple  belongs  to  the  true-result  or  the  maybe-result 
cf  the  query,  by  elimination  cf  alternative  cases.  The  algorithm 
is  net  new  and  it  may  not  be  always  applicable.  Cur  main 
contribution  is  the  formalization  of  the  problems  with  null 
values  in  the  framework  of  denotational  semantics.  This 
f erma 1 iza ticn  allows  a  better  understanding  of  these  problems  and 
gives  flexibility  for  "alternatives"  in  acceptable  semantic 
interpretations. 
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0  :  Tnt roduction 

The  DML  of  the  CODAS YL  COBOL  J OD  [ COD78 ]  is  widely 
criticized  for  being  too  low  level  to  be  used  in  the  developing 
areas  of  distributed  databases,  associative  database  machines, 
etc  [NBS79]*  This  paper  represents  the  current,  incomplete 
state  of  a  system  which  will  eventually  produce  a 
n on-n avlga t i onal ,  non— procedural  DML  for  use  with  COBASYL 
databases*  The  terms  "navigational"  and  "procedural"  lack  hard 
definitions  in  this  context*  The  DML  being  developed  here  has 
the  following  design  goal:  It  allows  the  application  programmer 
to  specify  his  intention  to  the  DBMS*  A  query  takes  the  form  of 
a  specification  of  the  attributes  of  a  set  of  data*  The  DBMS 
collects  those  data  having  the  specified  attributes*  The 
programmer  is  unconcerned  with  the  process  which  accomplishes 
the  collection* 
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Other  authors  have  developed;  systems  which  operate  on  and 

r 

return  collections  of  records  from  CODASYL  or  network-organized 
databases  [3RAD,  EARN,  McG,  NUL].  These  authors  have  for  the 
most  par*  neglected  to  provide  a  theoretical  foundation  for 
their  work  and,  perhaps  more  importantly,  the  systems  they  have 
devel oped  are  not  closed  with  respect  to  their  operations.  An 
algebraically  closed  system  can  be  extended  beyond  Its  uses  as  a 
D ML  to  provide,  among  o  +  hers,  a  view  description  (SUBSCHEMA) 
1 annuage  and  a  schema- to- sc hema  transformation  language  useful 
in  database  restructuring  applications.  Such  a  system  is  of 
theoretical  interest  in  its  own  right. 

For  any  subset  of  a  given  universe  there  is  a 
characteristic  function  which  takes  the  value  'true*  only  on 
those  elements  which  ere  members  of  the  subset.  .  The  relational 
calculus  of  Codd  can  be  seen  in  this  light  [ CODD72 ] :  an  alpha 
expression  is  a  characteristic  function;  the  universe  in  which 
It  Is  interpreted  is  the  cross  product  of  the  relations 
apoearing  in  range  expressions;  the  elements  of  the  subset  are 
the  tuples  of  the  cross-product  for  which  the  alpha  expression 
is  true.  An  hlnderance  to  applying  these  ideas  to  CODASYL 
databases  has  been  the  lack  of  an  idea  of  the  universe  and  its 
elements.  Both  are  defined  by  the  concept  "set  of  occurrences" 
in  this  work. 


Section  1  introduces  the  basic  concepts  of  the  theory  of 
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nets.  Section  2  introduces  the  subclass  of  COPASYL  nets  and 
some  operations  which  are  shown  to  be  closed  on  the  subclass. 
The  COPASYL  net  is  not  meant  to  represent  the  "COPASYL  data 
model*"  most  of  the  salient  features  of  that  model  are  ignored. 
(Among  the  missing  features  are  record  identifiers,  order  and 
ordering  keys,  membership  classes  (AUTOMATIC,  MANUAL,  etc.)  and 
many  others. )  The  central  structuring  mechanism,  the  CODASYL 
set,  is  focused  upon  and  abstracted  as  the  cset.  This  concept 
also  abstracts  the  AREA  notion,  which  is  an  1-n  relationship* 

Section  3  introduces  the  path  and  the  crucial  notion  of  the 
simple  path.  It  is  claimed  that  all  records  on  a  simple  path 
are  related  to  each  other  in  a  meaningful  way  and  +hat  only 
records  on  simple  paths  are  meaningfully  related.  Section  4 
introduces  the  set  of  occurrences,  which  is  an  idea  derived  from 
the  idea  of  the  simple  path.  Section  5  sketches  an  applied 
predicate  calculus  for  stating  characteristic  functions  cf 
subsets  of  occurrences.  Section  6  gives  a  function  for 
reforming  sets  of  occurrences  into  nets.  Section  7  explains  the 
concept  of  exhaustion,  which  is  an  attribute  of  sets  of 
occurrences.  Section  8  begins  to  show  ways  in  which  results  of 
prior  sections  can  be  combined.  Section  9  is  a  simple  example. 
Section  10  concentrates  on  what  remains  to  be  done,  which  is  a 


great  deal 
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1:  Objects  and  types 

Given  a  set  of  atomic  individuals  called  record s  and  a 
Tunc t ion  T  takinp  records  into  the  set  of  type  3.  By  convention 
we  write  T(r)  =  R. 

1 

Records  may  be  collected  into  subsets  called  assemblies  • 

The  function  T  1 5  enlarged  to  include  assemblies  in  its  domain 

under  the  condition  that,  Tor  records  r  ,  r  : 

1  2 

1  (  (r  ,  r  }  )  2  [T ( r  )t  T(r  ) } 

12  12 

Further  i+  is  assumed  for  assemblies  a  ,  a 


1  2 

a  =  a  —  T  ( a  )  =  T  {  a  ) 


12  1  2 

That  is,  T  is  functional  on  assemblies  even  though  inspection  of 

an  assembly  is  not  sufficient  to  determine  its  type.  This 

corresponds  to  the  intuitive  notion  of  type  as  describing  the 

potential  values  of  a  construct, 

A  cget  (pronounced  see-set)  is  a  pair  (r,  a)  where  r  is  a 

record  and  a  Is  an  assembly.  If  c  =  |r,a)  Is  a  cset  then  T(c )  - 

T( (r,a) )  =  (T(r),  T(a)).  The  record,  r,  is  called  the  owner  of 

c  ,  vr  itter  r  —  own(c)  ?  the  elements  of  the  assembly,  a,  are 

cal  led  the  mb ers  of  c,  written  a  =  mem(c)  •  The  t  enan t s  of  c 

are  the  owner  and  members  taken  together!  ten (c )  =  {own (cl)  U 

neu(c) .  The  assembly  may  be  empty  but  the  owner  record  must  be 

present  in  a  cset.  By  convention,  T(c)  =  C. 

A  collection  of  csets  is  called  a  net.  If  n  =  { c  ,  c  , 

1  2 


1  The  term  •'assembly"  is  taken  from  the  work  of  McGee  t  Me  G  ] 
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•  •  •  ,  c  ]  then  T(n|  =  T(  (c  ,  c  T  • • •  ,  c  3  )  3  (T(c  )  ,  T(c  )  , 

k  1  2  k  1  ?. 

T(c  )}•  (The  discussion  for  assembly  types  also  applies  here.) 


k 

T(n)  =  N  is  called  a  schema *  Note  that  a 
identical  to  a  net:  a  set  of  pairs,  each 
subset  of  individuals  and  an  individual, 
operations  and  properties  of  nets  will 
s  chemas • 


schema  is  formally 
pair  consisting  of  e. 
This  implies  that 

have  analogues  for 


A  net  n  is  a  subnet  of  the  net  n  if^ 

1 

( Vc  e  n  )  { •}  c  e  n)  F  (  c )  =  T(c  ) 

1  1  1 

G  own(c)  =  own (c  ) 

1 

G  nem(c)  =3  mem(c  5 

1 


Table  1  shows  that  +he  domain  and  range  of  +  tie 
each  consist  of  four  pairwise  disjoint  sets*  This 
allows,  for  example,  the  unambiguous  identification 
of  an  arbitrary  cset.  If  c  is  a  cset ,  c  =  (x, 
own(c)  iff  T(x)  is  a  record  type*  The  conventional 
representatives  of  each  class  of  object  and  type 


func  t ion  T 

c  onst  rue  t  ion 

of  the  owner 
y  3  t  +  hen  x  = 
notation  for 
is  also  given 


in  table  1 
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object  class  mapped  onto  type  class 


records;  r y  g, 
assemblies;  a»  m 
csets;  c  y  c 

1 

nets;  nt  n 

t 


record  types;  S 

assembly  types;  A,  M 
cset  types;  C,  C 

1 

schema  s;  K  ,  N 

1 


Table  1 


2;  COI'A.SYL  nets  and  operations 


A  ne+t  n»  is  a  CO  DA  S  Y  L  net  iff  for  c  ,  c  e  n  with  T(c  ) 

1  2  1 

T  (c  )  » 


1)  own(c  )  =  own  (c  )  -»  c  =  c 

1  2  12 

ii)  own(c  )  *  own ( c  ) 

J.  <w 

-*  mealc  )  fi  mem(c  )  =  0 

1  2 


We  note  in  passing:  that  before  the  appearance  of  the  78  JOD, 


might  have  written  under  the  same  conditions  for  c  T  c  that 

1  2 

is  a  CODASYL  net  iff 


ten{c  )  n  ten  (c  )  t  0  -*  c  —  c 

1  2  12 


we 

n 


The  appearance 


of  recursive  sets  in  the  CODASYI.  specifications 
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allows  a  record  to  be  the  owner  of  one  cset  of  a  given  type  and 

a  member  of  the  same  or  another  cset  of  the  same  type. 

Note  that  these  constraints  are  on  the  occurrence  level 

(the  extension);  the  schema  level  (intention)  is  unconstrained. 

Clearly  the  set  of  CODASYL  nets  is  not  closed  wrt  the  se+ 

operations  uniont  intersection  and  difference.  New  operations 

are  defined  which  operate  on  and  produce  CODASYI  nets.  These 

are  net  union,  U« ,  net  intersection,  I.,  and  net  difference,  — • 

.  In  the  following,  let  n  ,  n  be  nets;  let  match  be  an 

1  2 

abbreviation  for  the  expression 

(  ( -3  c  en,  c  en  )  T(c  )  =  T  (  c  )  =  T  (  c ) 

112  2  1  2 

own ( c  )  =  own ( c  )  =  own ( c) ) 

1  2 

This  expression  pairs  a  cset  from  each  of  the  nets  such  that 

both  are  of  a  given  type  and  have  a  given  owner. 

Nejt  intersection 

n  I.  n  =  (c  I  match  S 

1  2 

mera(c)  =  mem(c  )  n  .Hemic  )} 

1  2 


Net  union 


Let  C  be  a  cset  type.  Tien  for  nets  n  ,  n 

1  2 

X  (n  ,n  )  =  (r  |  (-}c  e  n  ,  c  e  n  ) 

C  1  2  112  2 

( T  ( c  )  =  T  ( c  )  =  C)  e 

1  2 

own(c  )  *  own(c  )  0 

1  2 


r  e  mem(c  )  n  mem(c  )3 

1  2 
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Elements  ol  V  (n  in  )  «  called  c  on  filets  y  appear  as  members  of  a 

C  1  2 

cset  of  +ype  C  in  each  of  n  and  n  which  csets  have  distinct 

1  2 

owners.  Then 


n  U.  n 
1  2 


=  (c  |  natc h 


(mem(c)  =  merafe  )  VJ  tnem(c  ) 

1  2 


X  (  n  t  n  )  ) 

T  ( c  )  1  2 


not  ma  tc b 


[  [  -Jc  ( c  e  n  }  or  ( c  e  n  )  ]  & 

3  3  1  3  2 

T(c)  =  T ( c  )  S 
3 

own (c )  =  own! c  )  S 

3 

tnetn(c)  =  mem (  c  )  —  X  (n  ,n  )] 

3  T  ( c )  1  2 


Net  differe  nc  e 


n  -  .  n 
1  2 


(c  I  ma  tc  h  -* 


( mem ( c )  =  mem(c  )  -  mem(c  ) )  & 

1  2 

not  match  —  ( -}c  e  n  )  c  =  c  ) 

3  1  3 


In  generaly  +  he  assembly  of  a  cset  In  the  result  of  a  net 
operation  is  the  result  of  the  corresponding  set  operation  on 

I 

the  assemblies  of  'matched'  csets  of  the  operands. 

Cl_alta  The  operations  U.y  I.y  -.y  are  well  defined  binary 


2  In  a  programming  language  it  would  be  better  to  define  n  U. 
n  =  error  If  X  (n  yn  )  is  non-empty  for  any  C.  1 

C  1  2 


2 
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operations  on  CODASYL  nets* 

Proof 

Tn  each  deflniton  the  type  and  tenants  ol  each  neaber  of 
the  result  are  uniquely  given.  This  is  enough  to  uniquely 
specify  the  result.  Therefore,  the  operations  are  well  defined 
on  the  class  of  nets.  It  remains  to  be  shown  that  the  subset  of 
CODASYL  nets  is  closed  under  the  operations. 

The  proof,  which  is  not  given  here,  proceeds  by  exhaustion 
to  show  that  if  the  result  of  an  operation  is  not  a  CODASYL  net 
then  the  operands  were  not  both  CODASYL  nets.  Obviously  the 
recognition  of  conflicts  is  crucial  to  the  proof  for  net  union. 
Any  approach  which  chooses  a  cs et  for  the  conflicts,  in  contrast 
to  removing  them  completely,  would  not  provide  an  associative, 
commutative  operator. 

The  following  observations  give  support  to  the  belief  that 
the  defined  operations  are  in  some  sense  natural: 

A  and  B  are  subnets  of  A  U.  B  if  and  only  if  X  (A,  B)  is 

C 

empty  for  all  C. 

AT.  B  is  a  sufcne+  of  A  and  of  B 


A 


B  is  a  subnet  of  A 


161 


NETS:  Operations  and  Logic 

?:  Paths 

A  sequence  o  =  r  ,  • •• ,  r  is  a  path  of  records  wi thin  a 

0  k 

net,  n,  iff  (Vi  0  <  i  <  k  )  {  j-c  e  n)[  (own(c)  =  r  £  r  e  mem(c)) 

i  i  -  1 

or  (own(c)  =  r  $  r  e  meia(c)  )].  That  is,  p  is  a  path  if  and 

i  —  l  i 

only  if  for  every  pair  of  adjacent  records  in  the  sequence, 

there  is  a  cset  in  the  net  for  which  one  of  the  pair  is  the 

owner  and  the  other  of  the  pair  is  a  member*  The  sequence  p*  = 

c  ,  ••*,  c  isa  pa  th  of  c  se  ts  iff  there  exists  a  path  of 

1  k 

records,  p,  such  that  for  all  j,  1  <  j  <  k,  c  satisfies  the 

j 

path  condition  for  the  pair  ( r  ,  r  )  in  p*  p  and  p*  are 

j-1  j 

called  associated  paths* 


(o 

t  Cm 

n-c 

Co  , Cm  ,m  })=c 

1  1 

i 

112 

{o 

»  Cm 

0 

II 

Co  ,  (m  ,m  )  )  =  c 

1  I 

2 

2  12 

p 

=  o  , 

m 

p  =  o  ,  m  ,  o 

1 

1 

112 

O  '  = 

c 

P 1  =  c  ,c 

1 

1  2 

p 

=  O  , 

m 

p  —  o  ,  m  ,  o 

1 

1 

1  2  2 

p  ’  = 

c 

p  *  =  c  ,  c 

o 

■J 

1 

ne  t 

1 

net  2 

Fifrure  1 
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For  any  pat  h  of  records  | csets)  there  nay  be  more  than  one 

associated  path  of  csets  (records).  For  example,  figure  1  gives 

two  nets  and  the  associated  paths.  (In  both  nets  it  is  required 

that  T(c  )  *  T(c  ))•  This  behaviour  can  occur  in  practice. 

1  2 

Consider  a  database  whose  records  represent  the  entitles  HOUSE 

or  PERSON  and  whose  csets  represent  the  relationships  O^NS  or 

LIVES— IN.  An  owner  occupied  house  creates  two  csets  of 

different  types  but  identical  contents,  as  in  net  1  of  Figure  1. 

Further  investigation  into  this  phenomenon  has  not  occurred. 

Two  seemingly  related  issues  of  greater  urgency  have  also 

not  been  deeply  investigated.  These  are  the  occurrence  of 

multiple  paths  between  record  types  In  the  schema  and  recursive 
3 

structures  «  Indeed  the  notion  of  recursive  structure  is  not 
clear  in  this  context  and  a  definition  is  needed.  In  the  bulk 
of  what  follows,  it  is  assumed  that  none  of  these  conditions 
arise  in  the  nets  under  study  unless  specifically  included. 

A  path,  p'  =  c  ,  c,  is  a  s imple  path  (of  csets)  iff 

1  k 

(Vi,  j  1  <  i,j  <  k)  i  t  J  implies  c  *  c  •  In  other  words,  no 

i  J 

cset  occurs  more  than  once  on  a  simple  path  of  csets.  A  path  of 

records  is  simple  if  it  has  an  associated  path  of  csets  which  is 

simple.  This  notion  is  crucial  to  the  rest  of  the  work. 

Consider  the  network  given  by  (c  ,  c  ,  c  }  where 

12  3 


3  The  concepts  path  of  records  and  paths  of  csets  have  obvious 
analogues  in  the  universe  of  types.  If  p  is  a  path  of  either 
sort,  p  =  x  ,  ...,  x  ,  then  T(p)  =  T(x  ),  ...,  T(x  ). 

Ok  Ok 


16: 


NETS:  Operation*?  and  Logic 


c  =  (r *  [s  »  s  ]  } 

1  t  2 


c  =  £t  t  {s  n 

?  i 

c  -  (u,  (s  ]) 

3  2 


Records  t  and  u  are  connected  by  the  path  of  records  t,  s  ,  rt 

1 

s  *  u .  This  path  is  not  simple  since  its  associated  path  of 

2 

csets  contains  c  twice*  From  this  we  deduce  that  t  is  not  an 

1 

attribute  of  u  (nor  u  of  t)  in  this  net.  This  work  is  not 
concerned  with  the  semantics  of  relationships  among  data  items. 
However*  it  maintains  that  there  is  no  semantically  meaningful 
relationship  between  a  pair  of  records  in  a  network  database 
unless  there  is  a  simple  path  between  them.  This  claim  is  false 
insofar  as  the  database  designer  is  not  obligated  to  realize  all 
semantically  valid  relationships  as  edges  in  the  network  schema 
nor  is  the  database  constrained  to  realize  all  the  semantically 
valid  relationships  available  in  the  schema  (if  OPTIONAL /MAN UAL 
is  allowed) •  However*  there  Is  a  semantically  valid 

relationship  between  a  pair  of  records  in  a  net*  which 
relationship  is  supported  by  the  structures  of  the  net,  if  and 
only  if  there  is  a  simple  path  between  them.  Other  semantically 
meaningful  relationships  cen  be  discussed  only  in  a  wider 


context • 


4:  Sets  of  Occurrences 
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Let  n  be  a  net  and  r  be  a  record.  The  set  of  occurrences 

of  the  net ,  n,  wrt  r,  denoted  O  (r),  is  given  by 

n 

0  (r)  =  (t  |  r  e  t  6 

n 

(Vr  ,r  ,r  )((r  etSr  et  -  £JP(r  ,r  ))  S 

12  3  1  2  12 

(  not(  r  e  t)  -»  ( -}r  e  t )  (not  SP(  r  ,r  )) 

3  4  3  4 

where  SP(r  ,r  )  is  true  iff  there  is  a  simple  path  between  r 
i  J  i 

and  r  •  The  record ,  rt  is  the  root  of  the  occurrence  set. 

J 

Intuitively,  an  element  of  0  (r)  is  a  collection  of 

n 

records,  containing  the  root  record,  r,  such  that  there  is  a 

meaningful  relationship  between  every  pair  of  records  in  the 
collection.  Furthermore,  each  collection  is  as  large  as  it  can 
be  made  without  violating  that  constraint.  In  the  sample  net  of 
figure  2, 

O  (r)  =  (  (r  ,  s  ,  t)  ,  (r  ,  s  ,  u)  } 

n  12 

An  analogous  concept  can  be  formed  for  schemas  by  substituting 

T(n) ,  T(r) ,  T(r  ) ,  etc  for  n,  r,  r  ,  etc,  in  the  definition. 

1  1 

The  collection  of  all  sets  of  occurrences  of  a  net  wrt  a 

record  type  is  denoted 

O  (P)  =  (O  (r)  |  E  =  T  ( r  )  3 

n  n 

R  is  called  the  root  type . 

5 '  Restricted  sets  of  occurrences;  characteristic  functions 


A  restricted  set  of  occurrences  is  merely  a  subset  of  a  set 
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of  occurrences.  Associated  vith  each  restricted  se+  is  a 
characteristic  function.  This  function  is  defined  on  the  set  of 
occurrences  and  has  the  value  'true*  for  a  given  element  iff 
that  element  is  in  the  restricted  subset.  I  will  now  describe 
an  applied  predicate  calculus  for  specifying,  characteristic 
f  line  1 1  ons. 


!5  .  I  Syntax 

Record  v  ar  tables  (r  t  r  »  «••)  take  records  as  values. 

1  2 

F ach  record  variable  Is  associated  with  a  record  type  such  that 

only  records  of  that  type  may  t e  values  of  the  variable. 

Predicate  symbol s  (p,  q,  p  ,  q  )  are  abstractions  of  the 

1  1 

comparisons  made  in  database  queries  (e.g.,  EMPLOY FF_SALAP Y  > 
20000).  Therefore  a  given  predicate  symbol  should  be  restricted 
to  record  variables  of  e  given  type  or  types.  However*  this 
would  serve  no  useful  purpose  end  would  increase  the  notational 
inconvenience.  Hence  T  choose  to  ignore  these  restrictions. 

The  standard  collection  of  quantifiers  { ¥,  -f )  ,  connectors 
(f>,  or,  not )  and  parentheses  are  used.  The  formulae  of  the 

language  are  formed  as  follows: 

i)  If  p  is  an  n-ary  predicate  symbol  and  r  ,  ...»  r  are 

1  n 

record  variables,  then  p(r  ,  ...,  r  )  is  an  a tomi c  formula . 

1  n 

i 1 ) I f  f  and  g  are  formulae  and  r  is  a  record  variable,  then 
f  G  g,  f  or  g,  f  -  g,  not  f,  ¥rf,  -}rf ,  (f  ) 


:4<a=340t 


NETS:  Operations  and  Logic 


166 


are  formulae. 

ii i (Nothing  else  is  a  formula.  (Except  that  we  will  see  in 
section  5.3  that  charact  er istic  functions  may  be  used  as 
oredlcates. ) 

A  formula*  P»  is  a  we  11  f  ormed  c  harac teri st  ic  function 

( WFCF  or  CF )  wrt  a  set  of  occurrences,  O  (E),  iff  the  t  ype  of 

n 


4 

each  record  variable  in  P  appears  in  T(C  (  3)  )  • 


If  P  is  a  WFCF 


( P )  then 

n 

O  (r)  |P  = 

(t 

1  *  e 

O  (r)  S  P( t)) 

n 

n 

0  (P) |?  = 

Co 

(  r )  |P 

|  T(r)  =  R} 

n  n 


3  •  “>  Interpretation 

A  formula  with  free  variables  may  be  thought  of  as  a 

function.  Let  P  be  a  formula  free  in  r  *  •••*  r  •  P(t)  is 

1  n 

interpreted  by  assigning  or  bind! ns  to  each  r  that  record  in  t 

i 


of  the  proper  type.  An  interpretation  of  the 


bouna  to  records  is  assumed  to  be  available. 


The  interpretation  of  the  symbols 


predicate  symbols 


or  *  -*  and  not  are 


4  The  function*  T*  has  not  been  formally  expanded  to  include 
sets  of  occurrences,  but  such  an  expansion  is  easily  conceived. 
The  subject  will  be  reexamined  In  the  section  on  Exhaustion. 

In  a  given  occurrence*  t*  there  may  be  more  than  one  record 
f  a  given  type.  It  is  easy  to  see  this  can  happen  only  if 
here  is  more  than  one  path  between  the  root  type  and  the 
ultiply  occurring  type*  a  condition  I  have  already  admi tted  to 
ot  having  fully  considered.  An  apparent  solution  is  to 
istinguish  occurrences  by  the  path  used  and  distinguish 
ariables  accordingly.  This  does  not  appear  satisfactory  for 
he  recursive  case  in  which  an  Indeterminate  number  of  records 
f  a  given  type  may  appear  In  any  element. 
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apparent*  The  quantifiers  are  interpreted  wrt  the  universe 

O  (r) •  Let  P  be  a  WFCF  for  O  ( r)  with  free  variables  s»  s  , 
n  n  1 

...»  s  •  Let  P*  be  P  with  the  variables  s  ,  • •• f  s  properly 

n  In 

bound  to  the  records  in  t  e  O  (r) .  P*  has  s  as  its  only 

n 

variable.  VsP(t)  is  true  iff  (V  t'  e  O  (r))  P* ( t • )  is  true. 

n 

The  existential  quantifier  has  a  similar  definition. 

If  s  is  a  variable  free  in  WFCF  P,  it  is  possible  that  s 
will  have  no  assignment  in  the  interpretation  of  P(t);  that  is, 
there  nay  be  no  record  of  the  type  associated  with  s  in  t.  s  is 
then  assigned  the  distinguished  value,  null .  There  i s  no  way  to 
distinguish  this  null  as  unknown  or  inc  on si  stent  [  VA  S  ]  or  any 
other  refinement.  If  such  a  distinction  is  made  it  must  be  trade 
by  the  predicate  itself.  Since  the  nature  of  this  distinction 
is  not  the  subject  of  this  investigation,  it  is  assumed  all 
predicates  are  defined  for  the  value  null . 

5.S  The  nesting  of  CF * s 

The  expression  0  (R)|P  may  be  considered  an  unary  predicate 

n 

on  the  record  type  P.  Th e  predicate  formula,  0  (R)|P  (r),  for 

n 

T(r)  =  P,  is  true  if  the  set  O  (r)|P  is  non-empty;  otherwise  it 

n 

is  false.  The  effect  is  to  allow  nested  specifications  of  CF's. 

The  definition  of  the  w el 1- f ornedne ss  of  CF's  used  in  this 

way  is  not  obvious.  Clearly  the  CF,  P,  above  must  be  well 

formed  wrt  O  ( R )  •  If  +he  expression  is  nested  within  a  CF  for  a 
n 
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set  of  occurrences!  O  (S) *  what  should  be  the  relationships 

n 

b  etween  n  and  n '  ,  T(n)  and  T ( n'  ) ,  T(0  (  R)  )  and  T( O  ( S ) ) ? 

n  n 

6.  Forming  Nets  from  Sets  of  Occurrences 

In  order  to  create  a  closed  system  which  uses 

characteristic.  functions  to  form  a  net  from  a  net,  we  need  a 
means  of  converting  a  restricted  set  of  occurrences  into  a  net. 
This  Is  given  by 

N  ( 0  (  r )  )  =  {c  |  He*  f  n)  T(c)  =  T(c')  8 

n 

own ( c )  =  oto(c')  5 

( V  s )  ( -}  t  e  O  (r))  own(c)  e  t  & 
n 

6 

[s  e  mem  (c)  «— *  (  s  e  t  S  s  e  mem(  c  *  )  ] 

C lai m  The  function,  N,  mapping  sets  of  occurrences  into  nets  is 
well  defined. 

Proof  Not  developed. 

The  reformation  of  a  net  from  a  set  of  occurrences 
presupposes  the  existence  of  a  net  to  be  used  as  a  model.  This 
definition  uses  the  net  from  which  the  occurrence  set  was 
originally  formed.  Whether  it  is  possible  to  reform  a  net 
without  the  use  of  a  model  or  with  a  different  model  has  not 
been  explored. 

6  This  definition  fails  to  be  satisfactory  if  there  is  more  than 
one  simple  path  from  own(c)  to  s.  c  may  then  not  be  on  the  path 
allowing  own(c)  and  s  to  appear  in  a  single  element  of  O  (r). 

n 
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The  definition  can  be  extended  as  follows 


N(C  ( R)  )  =  U.  0  (r ) 
n  n 

the  union  bei  np  taken  over  al  l  r  such  that  T(r)  =  R. 

7.  Exhaustion 

In  +he  theory  of  praphs,  a  graph  is  said  to  be  connected  if 

there  is  a  path  between  each  pair  of  vertices*  This  notion  will 

not  suffice  here  for  two  reasons.  First,  a  path  between  records 

in  a  net  may  be  non— simple;  secondly,  disconnected  nets  may  be 

intuitively  quite  well  behaved.  (For  example,  any  net  whose 

schema  is  {R , [S} )  will  be  disconnected  if  there  are  two  or  more 

occurrences  of  record  type  R.)  Exhaustion  is  a  property  of  sets 

of  occurrences  which  somewhat  resembles  connectivity. 

0  (r)  exhausts  a  if  every  record  appearing  in  n  appears  in 

n 

some  element  of  C  (r).  By  extension,  O  (R)  exhausts  n  if  every 


n 

record  in  n  appears 

In  the  last  example  of 

of  0  (s  ),  • . . ,  0  (u) 

n  1  n 

exhaustive  hut  neither 

(0  (s  ) ,  O  (s  ))  =  (  (  (r , 

n  1  n  2  1 

Extension  of  the  concept  to 

O  (R)  exhausts  N  if  every  rec ord 
N 


e leme  nt 

of 

O  (R) 
N 

.  There  are 

exhaust  ion 

i  n  the 

schema • 

Theor  era 

7. 

1  I  f 

O  ( P )  exhausts 

n 


element  of  O  (R). 

n 

is  exhaustive  but  none 

O  (  R)  ,  0  (  S  )  are 

n  n 

O(U)  are.  (0(S)  = 
n  n 

{{r,  s  ,  u)  )  )  )  • 

2 

the  schema  is  straightforward, 
type  in  N  appears  in  some 
some  interesting  results  about 


n,  then  O 

T  (  n  ) 


n 

in  some  element  of  some 

sect  ion  3,  O  ( r ) 
n 

are  exhaustive 

of  O  (T)  nor 
n 

,  t]3 


(R)  exhausts  T(n) 


170 


NETS:  Operations  and  Logic 

That  is,  exhaustion  of  the  schema  is  necessary  for  exhaustion  of 
t  he  net. 

Proof  Let  s  be  any  record  in  n.  T(s)  appears  in  T(n)  by 

definition  and  s  appears  in  0  (R)  by  assumption.  There  exists  p 

n 

=  r  ,  ...»  r  T(r  )  —  R  and  r  =  s  and  p  is  a  simple  path.  But 

0  k  0  k 

then  T(r  ),  ...»  T(r  )  foris  s  a  simple  pa  th  of  record  tyoes. 

0  k 

Therefore  there  is  a  simple  path  from  R  to  all  other  types  in 

T(n).  Therefore  O  (R)  exhausts  T(n).  end  of  proof 

T  ( n  ) 

A  forked  c  se  t  or  fork  has  a  type  (or  is  a  type)  with  more 
than  one  member  record  type. 

Theorem  7.2  If  R,  S  are  distinct  record  types,  N  a  schema  and  P 
=  P,  ...,  S,  a  path  between  F  and  S  in  N  containing  no  forks  in 

an  associated  path  of  csets,  then  there  exists  a  simple  path 

between  R  and  S. 

Proof  Choose  R,  S  in  N  and  P=R,...,  R,R  =  F,  R  =S,  the 

0  k  0  k 

shortest  path  without  forked  csets  from  R  to  S.  (P  need  not  be 

unique.)  We  will  show  that  if  P  is  not  simple  it  is  not  the 

shortest  path  without  forks  from  R  to  S. 

Assume  P  is  not  simple  and  le+  C  ,  ...,  C  he  an  associated 

1  k 

path  of  cset  types.  Choose  any  itJ  0  ^  i  <  j  ^  k,  such  that  C 

i 

=  C  .  By  definition  and  the  assumption  of  no  forked  csets, 

J 

{R  ,  P  )  =  ten( C  ) ;  (R  ,  R  )  =  ten(C  );  from  which  we  know 

i-1  i  i  j-1  J  J 

(R  ,  R  }  =  (R  ,  R  }  . 

1-1  i  J-1  J 

1  .  R  =  R  ;  R  =  R 

i  J  j-1  i-1 
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In  that  case  R  «  •  •  »  t  R  *  R  t  •  .  ,  ,  R  is  a  pa  th  fr  om  P  to 

0  i  j+1  k 

S  shorter  than  P  b3r  P  »  R  . 

i  +  1  J 

r  =  r  ;  e  =  k 
i  j-i  j  1-1 

Then  E  »  •••?  E  ?  R  »  •••»  H  is  a  path  from  P  to  S 

0  i~  2  j  k 

shorter  than  P  by  R  ,  .  ,,,  R 

1-1  j-1 

C  orol lary  7.2.1  If  N  is  a  schema  which  is  connected  and  has  no 

forked  sets,  then  for  all  record  types  R  in  Nt  O  ( E )  exhausts  N 

N 

and  is  a  singleton  set. 

Proof  Immediate  from  the  theorem. 

Coroll  ary  j[«2,2  If  N  is  a  connected  schema  without  forks  and 

R»  S  are  in  Nt  then  O  (R)  =  0  (  S)  for  all  Rt  S. 

N  N 

Proof  Immediate  from  Corollary  7,2,1 
e nd  of  proof 

Theorem  7,2  and  its  corollaries  tell  us  something  about 

WFCF '  s;  namely  if  T(n)  Is  connected  and  free  of  forks,  then  if 

the  variables  in  CF,  P*  reference  only  record  types  in  T(n) , 

then  P  is  well  formed  wrt  O  (R)  for  all  record  types*  R»  in 

n 

T  (n)  . 

The  next  +heorem  completes  exhaustion  for  schemas. 

Theorem  7,2  If  N  is  a  connected  schema  with  forks  then  O  ( R ) 

-  -  N 

exhausts  N  iff  there  is  a  simple  path  from  R  to  the  members  of 
all  forks, 

P  roo  f  Only  one  direction  is  difficult. 


Assume 


N  is  connected  and  there  is  a  simple  path  from  F  to 
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each  member  ol  a  fork.  We  must  show  that  there  is  a  simple  path 

to  all  other  record  types  as  well. 

For  record  type  S,  not  a  member  of  any  fork,  let  P  =  F  , 

0 

R  »  R  =  E  ;  R  =  S;  be  a  shortest  path  in  N  from  R  to  S. 
k  0  k 

If  it  contains  no  forks  then  apply  theorem  7.2.  Otherwise 

choose  i  such  that  P  =  p,  R  ,  ft  ,  q;  where  p  and  q  are  paths 

i-1  i 

according  to  the  algorithm: 

For  m  :=  k  by  -1  to  1  do 

If  C  is  a  fork  then  i  :=  ra ,  exit; 


m 

If  R  is  the  owner  of  C 


ra  m 

and  C  occurs  multiply  in  P 
m 

then  i  :=  m,  exit; 


e  nd ; 

One  of  these  conditions  must  occur.  In  both  cases  the  path  q  is 
simple.  I  will  show  that  P  non— simple  Implies  P  not  a  shortest 
path. 

1*  If  R  is  a  member  of  C  ,  necessarily  a  fork,  then  there 
i  i 

exists  p*  a  simple  path  from  R  to  R  *  If  p*»  q  is  simple 

i 

we  are  done.  If  not  it  must  be  the  case  that  multiply 
occurring  csets  occur  in  pairs,  one  from  p*  and  one  from  q, 
since  both  p*  and  q  are  simple.  Further,  since  q  contains 
no  forks,  no  multiply  occurring  csets  are  forks.  The 
reasoning  in  theorem  7.2  suffices  to  remove  at  least  one 


such  occurrence. 
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If  R  is  the  owner  of  the  f ork»  C  ,  then  there  must  exist 
i  i 

P*  »  a  simple  path  from  E  +o  R  •  If  the  path  p*,  C  ,  p  is 

i-1  i 

not  simple  then  either  C  occurs  multiply  or  it  does  not. 

i 

If  not t  then  no  multiply  occurring  c se t  is  a  fork  and  the 


reasoning  of  thercrem  7.2  again  applies;  if  so,  the 


reasoning  given  next  holds. 


If  R  owns  C  and  C  occurs 
i  i  i 

multiply 

in  P, 

then 

whe  ther  or 

not  C  is  a  fork,  k  , 

1  i 

the 

owne  r 

of  C 

i 

must 

also  occur 

multiply.  Select  j,  1.  <  j  < 

1 1 

so  that  E  = 

J 

R  . 

1 

Then  R  , 
0 

•  ••,  R  ,  R  t  •  ••,  R  is  a  path  eliminating  at  least  one 
j  i  +  1  k 


occurrence  of  C  • 


i 

end  of  proof 

This  theorem  points  out  a  degree  of  asymmetry  introduced  by 
the  use  of  forked  csets.  There  does  not  seem  to  be  any  simple 
means  of  characterizing  exhaustion  of  a  set  of  occurrences  in  a 
net  as  opposed  +o  a  schema. 

It  seems  reasonable  at  this  point  to  make  the  following 

C  on. fee  ture  N  (O  (R))  -  n  iff  O  (R)  exhausts  n. 

n  n 


8!  Combining  Nets 


Let  n  represent  t he  same  net  in  each  of  the  following 


e  xpressi ons ! 


(  *  )  N ( 0  ( R )  |P )  I .  N( O  (S)  |  Q) 

n  n 

(**)  N  ( O  (S)|0) 

~  NtO  ( R) | P) 
n 
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(***)  N(0  (  P  )  |  T  )  where 

n 

T  =  P  6  O  (  S)  |Q 
n 

These  expressions  represent  three  ways  of  combing  nets.  in 
(*)  two  nets  are  formed  independently  and  then  intersected.  In 
(*£)  an  intermediate  net  is  formed;  in  (***)  a  nested  Cl  is 
used.  A  complete  study  of  the  relationships  among  the  nets 
specified  by  ( * ) ,  (**),  and  (***  )  has  not  been  made.  Of 
particular  interest  are  the  conditions  under  which  these 
expressions  evaluate  to  the  same  net.  That  they  do  not  do  so  in 
general  has  been  discovered. 

In  set  theory,  if  S  has  characteristic  function  p  and  S' 
has  p'  then  S  n  S'  has  characteristic  function  p  R  p' •  Can  the 
CF's  of  (*) ,  (**) ,  and  be  expressed  in  terms  of  P  and  Q? 

P:  A  simple  example 

The  famous  Supplier-Parts  database  of  [DATE]  can  be 


represented  as  follows: 
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s: 

S#,  SNAME,  ... 

(  supp  Her  s  ) 

p: 

P#,  PNAME,  ... 

( parts) 

sp: 

OTY »  ... 

( June  t ion ) 

S - >  SP  < - P 


The  schema  of  this  net,  T(n)  =  ((S»  (3P]}*  (P»  (SP)  3  )  • 

Sets  of  occurrences  corresponding  to  the  query*  "Suppliers 

supplying  part  number  2”  can  be  specified  in  many  ways. 

1.  C  (S)  |  ¥P#  =  2 

n 

2.  O  (S)  |  -3-P#  =  n 

n 

3.  O  (S)  |  P#  =  ? 

n 

4 .  0  { P )  |  P#  =  2 

n 

The  first  set  contains  S,  SP,  P  triples  for  those  suppliers 
who  sell  only  part  #  2.  The  second  set  contains  a  set  of 
■triples  for  each  supplier  who  sells  at  least  part  #  2.  Each  set 
contains  a  triple  for  each  pari  sold  by  the  supplier.  The  third 
set  contains  the  same  number  of  elements  as  the  second  set*  but 
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each  element  contains  information  on  part  #  2  only  (each  element 
contains  a  single  triple  if  P#  is  an  " i dent i f i er "  for  P).  Set  4 
contains  a  single  element  within  which  is  a  triple  for  each 
supplier  supplying  pari  #  2.  N(3)  =  N(4)  which  are  subnets  of 
N(T)  and  supernets  of  N(l). 

The  query  "Suppliers  supplying  every  part"  does  not  have  an 
obvious  statement#  What  is  required  are  exactly  those  elements, 


s,  In  0  (S)  such  that  s  contains  all  the  parts  in  the  database, 
n 


10:  What  is  to  be  done? 

Beyond  the  need  to  solve  the  multiple  paths  and  recursion 
problems  and  the  need  to  derive  a  prograiaaing  data  sublanguage 
from  an  eventually  complete  abstract  presentation,  there  lie 
some  crucial  open  questions  previously  not  mentioned. 

The  set  functions  (AVO,  MIN,  etc) ,  called  Aggregates  by 
Stonebraker  [STON],  are  not  part  of  the  presentation.  The 
Aggregate  functions  of  Stonebraker  (the  GROUP  BY  of  3BQUFL) 
requires  a  subsetting  facility.  I  feel  these  will  not  prove 
d 1 f  f 1 cult • 


Allowing  a  navigational  DML  to  coexist  with  the  net— DML 
being  developed  here  will  require  additional  abstractions.  A 
navigation  takes  place  within  a  net.  The  details  remain  to  be 
worked  o\it  but  there  should  be  only  mechanics  to  be  considered. 

The  relational  model  allows  the  definition  cf  derived 
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relations*  To  al  lo»  a  notion  of  ’’derived  records"  requires 
considering  the  record  notion  as  a  complex,  not  atomic,  entity* 
Without  allowing  derived  records  it  is  difficult  to  perceive  how 
to  present  the  results  of  aggregate  functions  to  the  application 
program  as  data  values* 

Can  the  net— DML  he  used  for  the  construction  of  views  (as 
SEQUEL  is  used  in  System-F)  [SQL]  ? 

The  most  significant  task  yet  unaccomplished  is  the 
discovery  of  a  means  whereby  the  "goodness"  of  the  net— DMI  can 

be  established*  A  proof  of  its  completeness  viz  a  vis  the 

relational  calculus  requires  1)  a  notion  of  equivalence  for 

CCUASYL  and  relational  databases;  2)  a  function  mapping 
relational  DB’s  to  equivalent  CODASYL  DB'  sj  3)  a  function 

mapping  relational  calculus  expressions  to  net— DML  expressions; 
4)  a  proof  that  the  result  of  applying  the  net— DML  expression  is 
equivalent  to  the  result  of  applying  the  alpha  expression  to 
equivalent  databases*  Is  such  a  proof  necessary*  desirable  or 


possible? 
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1 .  Introduction 

Distributed  data  bases  are  receiving  much  attention  lately.  There  are, 
however,  some  reasons  to  retain  very  large  data  bases  centralized.  The  first 
reason  is  economic  advantages  through  centralization.  Large  data  bases  will  re¬ 
quire  large  capacity  storage  devices.  The  cost  per  bit  stored  decreases  with  the 
size  of  the  storage  device.  These  devices  are  and  will  probably  remain  expensive 
and  can  be  made  available  more  easily  through  centralization.  Centralization 
can  also  provide  users  with  much  needed  but  seldom  used  devices.  In  addition, 
there  are  some  technical  and  managerial  services  which  can  not  be  economical¬ 
ly  distributed.  The  second  reason  for  centralization  is  heavy  interaction  among 
users.  There  are  many  applications  where  users  interact  heavily  through  the 
data  base,  e.g.,  airline  reservations.  Large  scale  interaction  is  easier  to  handle 
through  cenralization.  Establishing  control  procedures  in  an  integrated  distri¬ 
buted  data  base  requires  close  coordination.  Accomplishing  coordination 
through  broadcasting  messages  widely  in  a  distributed  system  can  be  expensive. 

"“This  work  was  done  while  the  author  was  visiting  IBM’s  T.J.  Watson  Research 
Centre. 
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Let  us  safely  assume,  therefore,  that  there  "will  be  some  very  large  data  base  ap¬ 
plications  in  the  future  which  will  be  centralized. 

Large  data  bases  will  have  broad  use  and  we  can  expect  both  the 
number  of  users  and  the  number  of  transactions  to  increase  substantially.  In 
order  to  respond  to  these  needs  we  will  need  very  fast  processors  and  a  great 
amount  of  storage.  We  can  make  some  basic  assumptions  about  the  operation  of 
such  a  system.  All  the  assumptions  are  not  critical  for  our  subsequent  discus¬ 
sion,  but  they  establish  some  framework  to  express  our  ideas.  We  start  by  pos¬ 
tulating  the  existence  of  four  levels  of  memory  (Figure  1).  First,  there  is  a  bulk 
memory  consisting  probably  of  rotating  devices  with  great  capacity  but  rather 
low  speed.  Second,  there  is  a  system  memory  which  is  probably  based  on  CCD’s 
or  bubbles,  with  adequate  capacity  and  higher  speed.  Third,  there  is  a  processor 
memory  which  is  probably  monolithic,  fast  but  small.  Finally,  there  is  a  buffer 
memory  (cache)  which  for  the  purposes  of  our  discussion  will  be  considered  as 
an  integral  part  of  the  processor.  The  unit  of  system  memory  management  will 
be  called  a  segment.  Segments  are  transferred  between  bulk  memory  and  sys¬ 
tem  memory.  Segments  do  not  necessarily  correspond  to  IMS  segments  or  "seg¬ 
ment  table"  segments.  The  units  of  processor  memory  management  are  pages. 
Pages  are  transferred  between  system  memory  and  processor  memory. 

Consider  a  stream  of  on  line  transactions.  The  processor  will  inevitably 
access  pages  not  present  in  processor  memory.  Some  of  these  pages  will  be  in 
system  memory  and  some  pages  will  be  in  segments  in  bulk  memory.  The 
discrepancy  between  the  speeds  of  the  processor,  system  and  bulk  memories 
can  be  handled  using  a  high  level  of  multiprogramming  etnd  doing  task  switching. 
As  the  processor  speed  increases,  however,  the  processor  can  execute  the  in¬ 
structions  between  page  faults  very  quickly.  It  becomes  necessary  to  have  very 
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high  levels  of  multiprogramming  to  keep  the  processor  busy.  In  that  case  the 
overhead  of  task  switching  may  become  considerable.  We  should,  therefore, 
look  for  methods  to  reduce  page  faults  and  avoid  extensive  task  switching.  In 
this  way  we  can  achieve  high  processor  utilization  with  lower  levels  of  multipro¬ 
gramming. 

2.  Data  Locality 

We  anticipate  that  the  system  memory  will  be  large  enough  to  hold  a 
substantial  portion  of  the  operating  system  ans  user  programs.  On  the  other 
hand,  the  system  memory  can  only  hold  small  parts  of  a  data  base  at  any  in¬ 
stant.  We  do  not  have,  therefore,  a  staging  problem  for  program  segments 
between  bulk  memory  and  system  memory.  We  have  a  staging  problem  mainly 
for  data  base  segments.  We  also  expect  that  pagination  of  programs  between 
system  and  processor  memory  can  be  handled  with  techniques  similar  to  the 
ones  used  today  between  main  and  secondary  storage.  We  will  concentrate, 
therefore,  on  paging  techniques  for  staging  the  data  base.  All  paging  and  seg¬ 
mentation  techniques  for  programs  are  based  on  locality.  Since  we  want  to  con¬ 
centrate  on  data  base  segmentation  and  paging  we  need  to  explore  a  notion  of 
data  base  locality. 

Let  us  first  try  to  visualize  the  reasons  why  programs  exhibit  locality  of 
reference  and  then  try  to  interpret  our  observations  in  a  data  base  context. 
Consider  a  person  writing  a  program.  It  is  good  programming  style  to  put  adja¬ 
cent  in  the  program  the  parts  which  are  logically  related.  Moreover,  the  pro¬ 
gram  is  assumed  to  execute  its  statements  sequentially.  There  are  logical 
discontinuities  in  control,  but  they  are  infrequent.  The  compiler  compiles  the 
program  sequentially  and  emits  code  in  such  a  way  that,  after  loading,  the 
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sequentiality  is  to  a  great  extent  retained.  When  the  program  is  executing,  its 
references  follow  an  overall  sequential  pattern.  Locality  of  reference  is,  there¬ 
fore,  a  direct  result  of  the  sequential  nature  of  the  program.  When  we  carefully 
block  a  program  into  pages,  small  jumps  which  appear  often,  e.g.,  tight  loops, 
are  usually  subsumed  in  the  same  page.  Hence,  although  the  strict  sequentiality 
of  the  program  is  broken,  sequentiality  on  the  more  macroscopic  page  level  is 
retained  and  locality  is  enhanced. 

Sequentiality  by  itself  would  probably  imply  a  FIFO  policy  with  some 
prefetching.  In  this  way  we  can  retain  a  window  of  memory  which  the  program 
would  probably  reference.  However,  there  are  other  factors.  First  we  can  ac¬ 
complish  prefetching  without  doing  prepaging.  There  are  so  many  instructions 
in  a  page  that  a  sequential  program  can  execute  for  quite  a  while  before  it  needs 
another  page.  Demand  paging  does  some  prefetching  and  is,  therefore,  accept¬ 
able.  A  FIFO  policy  with  a  small  number  of  pages  will  be  sufficient  for  a  com¬ 
pletely  sequential  program.  Programs,  however,  are  not  completely  sequentiaL 
As  a  matter  of  fact,  they  usually  come  back  often  to  the  same  place.  This  is  be¬ 
cause  some  small  procedures  or  control  blocks  are  so  basic  that  they  get  refer¬ 
enced  frequently.  An  LRU  policy  helps  in  keeping  the  pages  of  these  often  refer¬ 
enced  parts  in  memory.  Hence  LRU  policies  do  rather  well. 

The  locality  of  program  references  is  based  on  sequentiality.  Data 
bases  are  not  sequential.  Their  logical  structure  is  not  always  linear.  Data  bases 
do  not  always  get  accessed  in  a  manner  which  corresponds  logically  to  a  sequen¬ 
tial  pattern.  Only  in  certain  cases  the  data  base  logical  structure  is  traversed  in 
a  sequential  way.  That  is  exactly  what  IMS  does  by  using  the  command  GET 
NEXT  in  the  data  base  trees  [Date].  The  system  is  supposed  to  store  the  data  in 
a  manner  consistent  with  the  sequential  logical  traversal.  In  this  way  the 
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sequentiality  of  the  stored  data  can  give  a  reasonable  locality  of  references  for 
the  sequential  logical  data  traversal.  This  approach,  however,  has  many  prob¬ 
lems.  The  data  base  is  assumed  to  be  predominantly  traversed  in  one  sequential 
way.  Other  sequential,  or  even  non  sequential,  traversals  are  precluded.  When 
other  data  base  traversals  are  effected  in  terms  of  the  one  predominant  sequen¬ 
tial  traversal  many  records  are  skipped.  As  a  result,  the  locality  suffers.  Final¬ 
ly,  managing  the  memory  to  implement  this  forced  sequentiality  is  difficult. 

If  we  assume  that  the  navigation  is  according  to  data  base  structure, 
then  locality  should  be  defined  accordingly.  When  we  look  at  a  particular  record 
the  records  within  the  locality  should  be  the  ones  which  are  close  in  terms  of  the 
data  base  structure.  For  instance,  in  a  hierarchical  data  base  this  means  the 
ancestor  records,  the  immediate  brother  records  and  the  descendent  records. 

In  a  navigation  through  a  hierarchical  or  network  data  base  we  need  to 
keep  a  reference  window  of  data  which  are  close  to  the  data  we  are  currently 
looking  at.  The  data  we  should  replace  are  the  furthest  out  in  our  navigation.  In 
a  navigation  we  do  not  always  come  back  to  the  same  data.  As  a  matter  of  fact, 
once  we  see  some  data  we  can  keep  it  in  memory  and  there  is  no  reason  to  visit 
it  again  unless  we  want  to  replace  it.  The  system  does  keep  some  currency  indi¬ 
cators  which  are  important.  If  we  backtrack  that  is  where  we  will  probably  go. 
Hence  some  special  provision  for  pages  referenced  by  currency  indicators 
seems  a  reasonable  choice. 

There  is  an  observation  in  data  bases  that  80%  of  the  accesses  deal  with 
20%  of  the  data.  This  situation  seems  to  point  to  an  LRU  replacement  policy.  It 
is  not  realistic  to  page  in  such  a  large  portion  as  20%  of  the  data  base  in  the  pro¬ 
cessor  memory.  LRU  is,  however,  a  good  policy  for  bringing  data  base  segments 
from  bulk  memory  into  system  memory.  An  LRU  policy  is  also  important  for 
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paging  into  processor  memory  data  description  tables,  DBMS  programs  and 
small  portions  of  data  which  are  referenced  very  often.  We  expect  portions  of 
programs  and  system  tables  to  be  almost  permanently  resident  in  processor 
memory.  Our  discussion  points  to  fetching  strategies  based  on  a  notion  of  locali¬ 
ty  provided  by  the  data  base  structure.  It  also  points  to  replacement  strategies 
based  on  LRU. 

In  program  demand  paging  we  accomplished  prefetching  without 
prepaging,  because  there  were  so  many  instructions  per  page.  Data  base 
records  can  be  large.  In  order  to  prefetch  with  demand  paging  we  need  large 
blocks  as  pages  [Ragaz  and  Rodriguez-Rosell].  Alternatively,  we  can  prefetch 
pages  [Smith].  To  prepage  we  need  to  know  what  pages  will  be  accessed  next 
with  high  probability. 

3.  Non-Linear  Virtual  Address  Space 

Consider  a  process  executing.  We  would  like  to  determine  the  working 
set  of  pages  that  it  requires  to  proceed.  In  terms  of  programs  the  working  set 
traditionally  meant  the  pages  that  the  program  has  seen  lately.  It  was  predicted 
that  the  program  would  need  the  same  pages  in  the  near  future.  In  terms  of 
data  the  working  set  does  not  necessarily  correspond  to  the  part  of  the  data 
that  the  program  has  seen  before.  It  should  correspond  to  the  data  that  are 
"close"  to  what  the  program  sees  now.  "Close"  is  defined  according  to  the  struc¬ 
ture  of  the  data  and  the  way  that  it  is  being  accessed,  i.e.,  by  the  notion  of  local¬ 
ity  of  data. 

We  already  have  an  elementary  notion  of  closeness  in  pages  by  group¬ 
ing  them  into  segments.  Pages  in  the  same  segment  are  logically  related.  Seg¬ 
ments,  however,  are  rather  large  and  we  can  not  afford  to  have  all  the  pages  of  a 
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given  segment  resident  in  processor  memory.  We  need  a  more  refined  notion  of 
locality  which  can  structure  pages  within  a  segment.  In  this  way  a  particular 
page  will  determine  other  pages  which  are  logically  close  to  it.  Two  traditional 
ways  of  linking  blocks  of  storage  can  be  used  to  thread  together  pages.  In  one 
approach  there  may  be  a  separate  structure  which  tells  the  system  what  pages 
are  close.  To  follow  this  approach  means  that  the  page  tables  will  have  to  be 
modified  and  augmented  to  include  extra  information.  The  page  tables,  howev¬ 
er,  are  supposed  to  provide  the  mapping  to  physical  addresses  in  a  nice  and  sim¬ 
ple  way  and  giving  them  added  structure  may  be  unacceptable.  In  a  second  ap¬ 
proach,  a  page  may  contain  information  about  the  pages  to  which  it  is  related. 
This  would  imply  that  a  portion  of  the  page  is  used  for  system  purposes,  which  is 
rather  unnatural. 

We  propose  a  third  approach,  in  which  the  virtual  address  space  itself 
is  structured.  In  this  way  the  address  of  a  page  will  reveal  addresses  of  other 
pages  to  which  it  is  related.  The  virtual  space  up  to  now  has  been  considered 
linear,  at  least  for  pages  within  segments.  It  does  not  have  to  be  linear.  Since  it 
is  virtual  it  can  be  structured  in  any  one  of  many  other  ways.  It  can  be  typed  to 
be  an  array  of  pages,  or  a  hierarchy  of  pages,  etc. 

Suppose  we  have  a  generalized  address  consisting  of  segment  id,  then 
page  id,  then  word  id.  The  word  id  gives  the  address  within  a  page.  The  segment 
id  identifies  a  particular  segment.  The  segment  id  is  augmented  to  declare  a 
segment  to  be  of  a  particular  type.  The  type  can  be  linear,  two-dimensional,  n- 
dimensional,  hierarchical,  etc.  This  means  that  the  pages  within  the  segment 
are  structured  accordingly.  The  page  id  is  traditionally  a  number  identifying  a 
page  within  the  segment.  We  choose  to  interpret  the  number  as  an  n-tuple 
alt  a&  ....  an.  This  interpretation  is  obtained  either  by  decomposing  the  page 
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id  into  groups  of  digits,  or  by  encoding  n-tuples  into  one  number.  By  using  en¬ 
codings  of  n-tuples  into  a  page  number  we  can  deal,  up  to  a  certain  point,  with 
tuples  of  varying  n. 

Consider  as  an  example  the  page  allocation  of  a  large  two  dimensional 
array.  It  is  allocated  into  a  segment  of  type  2-dimensional.  The  means  that  all 
page  addresses  within  the  segment  are  treated  as  2-tuples  ( n,m ).  Suppose  that 
the  system  is  clever  enough  to  allocate  the  array  in  the  matrix  of  pages  in  the 
natural  way.  That  is,  the  array  is  blocked  into  pages  and  it  is  put  within  the  ma¬ 
trix  of  pages.  Within  a  page  the  array  can  be  allocated  in  any  desirable  way.  As 
a  result  of  this  allocation  the  parts  of  the  array  which  are  logically  close  have 
close  page  addresses.  Close  page  addresses  are  obviously  (n.m),  (n',m')  with  n- 
n'  and  m-rrC  small.  The  system  can  use  the  information  to  physically  allocate 
the  pages.  It  can  try  to  place  the  pages  which  are  logically  close  physically 
near.  In  addition,  the  memory  manager  can  prepage  if  it  chooses.  For  instance, 
if  the  machine  has  vector  hardware  it  can  establish  a  pipeline  of  pages  as  they 
will  probably  be  needed.  Notice  that  there  is  still  uncertainty  about  how  one 
would  access  the  array.  Arbitrary  accesses  which  do  not  follow  any  pattern  in 
the  array  will  generate  a  random  pattern  of  page  accesses  and  many  page 
faults.  A  natural  progression  within  the  array  according  to  rows,  columns,  or  di¬ 
agonal  will  generate  a  set  of  pages  which  are  close,  i.e.,  they  have  close  page  ad¬ 
dresses. 


The  virtual  space  can  be  organized  in  other  than  array  structures.  For 
instance,  we  can  have  a  hierarchy  of  pages.  Suppose  we  have  a  hierarchical  data 
base  structure.  We  can  block  the  nodes  and  allocate  them  in  pages.  Each  page 
will  have  an  n-tuple  as  an  address.  Moreover  the  address  of  each  page  will  iden¬ 
tify  the  address  of  pages  which  have  nodes  close  in  the  tree.  The  technique 
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bears  resemblance  to  a  trace  Implementation  of  hierarchical  systems  [Tsi- 
chritzis  and  Lochovsky].  The  case  of  network  structures  is  much  more  compli¬ 
cated.  The  scheme  does  not  seem  to  generalize  easily  for  networks.  We  can 
force  users  to  operate  always  out  of  a  hierarchical  subschema.  This,  however, 
will  necessitate  that  the  segments  be  associated  with  subschemas  and  that  the 
binding  of  the  data  to  a  segment  type  be  done  during  loading. 

It  should  be  obvious  that  we  have  some  overhead  with  this  method. 
First,  we  need  bigger  page  addresses  because  we  do  not  use  them  compactly. 
For  instance,  in  a  particular  organization  some  page  addresses  may  not  be  used. 
This  necessitates  a  larger  page  address.  Notice  that  if  we  use  encoding,  then  we 
can  pack  the  page  addresses  better  than  if  we  decompose  the  page  address  into 
groups  to  get  n-tuples.  In  addition,  encoding  of  n-tuples  does  not  necessitate 
any  change  in  the  hardware.  We  will  just  have  to  change  the  interpretation  of 
page  addresses  and  view  them  as  n-tuples  after  we  decode  them.  A  second 
disadvantage  is  that  the  part  of  the  system  which  allocates  virtual  address  space 
is  more  complicated.  It  does  not  just  get  a  series  of  pages.  It  has  to  structure 
the  pages  and  distribute  the  data  appropriately.  This  requires  some  overhead. 
However,  if  the  loading  into  virtual  space  is  not  done  often  compared  with  ac¬ 
cessing  then  it  may  be  acceptable. 

Careful  analysis  and  experimentation  is  needed  to  establish  the  feasi¬ 
bility  of  this  approach.  Consideration  should  be  given  to  the  fact  that  the  chan¬ 
nels  of  communication  between  the  different  memories  are  congested  and 
prepaging  may  worsen  the  bottleneck.  The  scheme,  however,  should  be  regard¬ 
ed  mainly  as  a  means  of  presenting  the  memory  manager  with  additional  infor¬ 
mation  about  expected  reference  behavior.  If  and  how  this  information  will  be 
used  to  prepage  will  depend  on  the  status  of  the  overall  system  and  the  traffic  in 
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the  channels.  There  is  no  need  to  guarantee  that  needed  pages  will  be  in  the 
processor  memory.  On  a  statistical  basis  we  are  trying  to  cut  down  on  page 
faults  and  task  switching  by  doing  some  prepaging. 

4.  Transaction  Pipeline 

In  the  previous  section  we  looked  at  paging  and  at  ways  of  decreasing 
overall  data  base  page  faults.  The  segment  traffic  between  bulk  and  system 
memory  is  different.  A  segment  fault  will  take  a  long  time  to  be  served.  During 
this  time  the  process  will  be  completely  inactive  and  will  be  a  dead  burden  to 
our  level  of  multiprogramming.  It  would  be  nice  if  we  could  predict  the  seg¬ 
ments  needed  for  a  transaction  and  bring  them,  in  whole  or  in  part,  into  system 
memory.  Before,  however,  looking  at  segmentation  we  should  get  an  idea  about 
how  transactions  get  serviced  so  far  as  the  data  base  is  concerned. 

Consider  a  large  system  servicing  a  series  of  transactions  on  a  large 
data  base.  In  an  over  simplification,  the  data  base  operations  can  be  thought  of 
as  a  sequence  of  (SELECT,  UPDATE)  pairs.  There  are  other  operations  or  actions 
that  the  programs  take  but  we  will  not  consider  them. 

The  operation  SELECT  refers  to  isolating  a  part  of  the  data  base,  usual¬ 
ly  a  record,  which  we  want  to  access.  UPDATE  means  that  the  selected  record  is 
changed  in  some  way.  UPDATE  is  optional.  Any  UPDATE,  however,  is  preceded 
by  a  SELECT.  After  the  SELECT  operation  most  systems  put  the  selected 
record(s)  in  a  hold.  The  memory  manager  knows  that  the  record  will  change 
and  keeps  the  appropriate  data  base  part  close  to  the  processor.  The  important 
part  for  the  management  of  the  memory  hierarchy  is  the  SELECT  operation.  We 
can,  therefore,  concentrate  without  loss  of  generality  on  the  data  base  accesses 
as  a  series  of  SELECT  operations. 
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The  selection  operations  in  almost  every  data  base  management  sys¬ 
tem  are  of  two  kinds.  One  kind  is  a  selection  "out-of-the  blue"  which  means 
select  a  record  or  part  of  the  data  base  on  the  basis  of  contents.  A  second  kind 
is  selection  "relative  to-where-I-was-before"  which  means  that  the  system  uses 
currency  indicators  to  find  where  it  was  and  navigates  in  the  data  base  accord¬ 
ing  to  the  data  base  structure.  We  will  denote  as  Sc  (select  on  content)  and  Sn 
(select  by  navigation)  the  two  types  of  SELECT  operations  respectively.  Process¬ 
ing  usually  begins  with  an  Sc  selection  followed  by  a  series  of  Sn  operations.  For 
example,  GET  UNIQUE  in  IMS  positions  in  the  right  data  base  tree  and  then  GET 
NEXT  traverses  the  tree  [Date,  Tsichritzis  and  Lochovsky]. 

The  Sn  operations  are  rather  nice  as  far  as  the  memory  hierarchy  is 
concerned.  They  can  obviously  skip  records  according  to  type  or  contents. 
However,  they  usually  pick  records  which  are  logically  close  in  the  data  base. 
Logically  close  records  should  be  allocated  physically  near.  A  buffering  scheme 
based  on  sequentiality  will,  therefore,  work  nicely  and  we  will  have  the  right  part 
of  the  data  base  close  to  the  processor.  The  Sc  selections  behave  quite 
differently.  They  jump  to  a  part  of  the  data  base  which  has  nothing  to  do  with 
what  was  happening  before.  However,  Sc  selections  use  mainly  indexes.  Indexes 
are  traversed  using  navigation  which  can  be  exploited  by  a  notion  of  data  locali¬ 
ty  on  the  index. 

If  we  have  an  environment  with  many  Sn  operations  per  Sc  operation 
then  the  system  will  do  well  with  a  buffering  scheme  for  the  data  base  based  in 
data  locality.  This  is  the  case  in  a  batch  data  base  operation  where  large  parts 
of  the  data  base  are  traversed.  Unfortunately,  in  an  on-line  transaction  oriented 
system  it  may  not  be  the  case.  Each  transaction  will  probably  be  a  short  burst 
of  one  Sc  selection  followed  perhaps  by  a  few  Sn  selections.  In  such  a  situation 
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the  processor  will  pick  up  the  transaction  and  the  transaction  will  ask  for  its 
data  according  to  the  Sc  selection.  The  indexes  will  have  nice  data  locality  pro¬ 
perties.  The  data  in  the  data  base,  however,  may  not  even  be  in  system 
memory.  The  system  will  have  to  put  the  transaction  in  a  waiting  state  until  the 
data  is  brought  in.  In  a  situation  like  this  many  transactions  will  be  in  a  wait 
stage.  Task  switching  will  be  frequent,  and  hence,  expensive.  The  processor 
essentially  looks  at  each  transaction  twice:  once,  to  figure  out  what  data  it  wants 
and  ask  for  it,  and  second  to  service  the  transaction.  If  the  above  scenario  is 
realistic  we  need  to  take  it  into  account  in  both  memory  management  and  pro¬ 
cessor  scheduling. 

In  one  scheme  we  can  have  the  system  realize  that  an  Sc  selection  may 
necessitate  bringing  in  a  segment  from  bulk  memory.  The  processor  not  only 
task  switches  but  also  puts  the  task  in  a  dormant  state  for  a  while.  In  another 
scheme  we  can  try  to  prestage  the  appropriate  data.  A  front-end  processor  can 
look  inside  the  transaction.  It  can  pass  some  information  to  the  memory 
manager  about  Sc  selections,  which  the  memory  manager  may  use  to  pre-stage 
the  right  part  of  the  data  base  before  the  processor  even  touches  the  transac¬ 
tion.  This  will  obviously  work  only  in  the  case  that  the  Sc  selection  is  explicit, 
i.e.,  the  content  is  specified  as  a  constant  and  not  as  a  program  variable.  Isolat¬ 
ing  the  Sc  selections  is  not  that  farfetched.  Most  data  base  systems  have 
separate  commands  for  this  kind  of  selection. 

With  this  scheme  we  hope  to  establish  a  pipeline  of  transactions,  and 
stage  the  data  in  the  different  parts  of  the  memory  hierarchy  accordingly.  As 
far  as  the  data  base  is  concerned,  the  accesses  can  be  thought  of  as  analogous 
to  a  pipeline  of  instructions,  with  Sc  playing  a  role  similar  to  jumps  and  Sn  play¬ 
ing  a  role  similar  to  sequential  execution  of  machine  instructions. 
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5.  Segmentation  of  Data  Bases 

Consider  a  segment  as  a  unit  of  management  and  transfer  between 
bulk  and  system  memory.  There  are  some  fundamental  decisions  to  be  made 
regarding  segments.  The  first  question  is  whether  the  segments  will  be  typed  or 
just  be  bags  to  put  data  in.  The  second  question  is  whether  or  not  the  segments 
will  correspond  to  data  base  types  or  data  base  areas  in  the  data  base  definition. 
Finally,  there  is  the  decision  about  how  the  segments  will  be  transfered  in  whole 
or  in  parts  and  how  they  will  be  managed. 

To  illustrate  the  issues,  consider  a  hierarchical  data  base.  A  data  base 
tree  consists  of  a  root  record  occurrence  and  all  the  record  occurrences  des¬ 
cendant  from  it.  Typically,  hierarchical  systems  associate  a  data  base  tree  with 
one  block.  They  then  associate  a  file  with  all  blocks  of  data  base  trees  according 
to  a  hierarchical  definition  tree.  In  this  example  we  have  two  straightforward 
approaches.  In  one  approach  we  can  call  a  segment  a  block  as  it  corresponds  to 
a  data  base  tree.  In  the  second  approach  we  can  associate  a  segment  with  the 
whole  file  corresponding  to  all  the  occurrences  in  a  hierarchical  definition  tree. 

The  first  approach  is  what  hierarchical  systems  do  now  by  buffering 
blocks.  In  our  environment  there  will  be  some  difficulties.  First  and  foremost, 
system  memory  is  much  bigger  than  current  main  memories.  The  data  base 
tree,  therefore,  will  be  a  rather  small  unit.  We  can  buffer  many  more  in  and  use 
the  memory,  but  they  will  probably  not  correspond  to  what  we  need.  We  will 
have  difficulties  with  selections  which  move  us  to  a  different  tree.  Finally,  the 
segments  will  not  have  a  uniform  type. 
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In  the  second  approach  we  treat  the  whole  data  base  as  a  segment. 
This  is  the  opposite  extreme.  The  segments  will  be  too  large.  We  can  not  afford 
to  keep  in  system  memory  all  the  data  associated  with  a  definition  tree. 

We  need  a  segment  unit  which  has  size  between  the  data  base  tree  and 
the  whole  data  base.  We  need  a  notion  of  data  base  area  which  is  treated  as  a 
segment.  We  can  obtain  this  area  by  partitioning  the  data  base  either  according 
to  occurrences,  or  according  to  types.  Partitioning  in  terms  of  occurrences 
would  mean  that  we  group  some  data  base  trees  or  other  occurrences  on  the 
basis  of  contents.  Partitioning  in  terms  of  types  means  that  we  use  different 
record  types,  attributes,  or  aggregations  of  them  for  defining  the  segment  units. 
We  prefer  partitioning  in  terms  of  types  because  it  is  much  cleaner.  The 
corresponding  segments  are  nicely  typed.  Finally,  it  is  much  easier  to  predict 
what  segments  a  program  or  transaction  will  need. 

We  will  outline  an  example  using  segmentation  according  to  data  base 
types.  Consider  a  data  base  consisting  of  record  types  R  1,R2,...,Rtu  Among  the 
record  types  there  are  known  connections,  or  links,  Ll,L2,...,Lk.  These  links 
correspond  to  parent  child  relationships  in  hierarchical  systems,  DBTG  sets  in 
network  systems,  etc.  Let  us  assume  that  there  are  also  indexes  /  1,72... .,/£.  Let 
us  suppose  that  the  record  types,  the  links  and  the  indexes  are  implemented 
separately.  We  define  a  segment  for  each  R,L  and  /.  So  we  bring  in  and  out  of 
system  memory  segments  corresponding  to  record  types,  links,  and  indexes.  As 
a  first  observation  we  see  that  the  R,L,I  's  will  correspond  to  uniformly  typed 
segments.  They  are  not  as  small  as  a  data  base  tree,  nor  as  large  as  a  whole 
data  base.  As  a  second  observation,  we  can  easily  predict  what  segments  a  tran¬ 
saction  may  access.  First,  we  pick  all  the  record  type  segments  which 
correspond  to  data  base  variables  that  a  program  accesses.  Second,  we  pick  all 
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the  index  segments  for  attributes  appearing  in  content  selections.  Third,  we 
pick  all  the  link  segments  corresponding  to  navigation  through  the  data  base.  If 
the  navigation  is  not  explicit,  we  can  still  figure  out  what  links  the  system  will 
use  to  go  from  a  given  record  type  to  another  record  type.  Intermediate  record 
type  segments  are  brought  in  memory  in  the  case  of  implicit  navigation,  e.g. 
when  going  up  and  down  a  hierarchical  data  base.  Notice,  that  this  set  of  seg¬ 
ments  is  not  exactly  what  the  program  will  access  but  the  segments  that  the 
program  may  possibly  access. 

It  might  be  nice  for  transactions  to  declare  what  segments  they  will 
need,  using  an  appropriate  control  language.  Declaration  of  views  and  subsche¬ 
mas  may  actually  already  provide  most  of  the  needed  information.  In  that  case 
there  is  no  need  for  preprocessing  and  the  memory  manager  can  bring  the 
needed  segments  in  ahead  of  time.  However,  it  will  be  hard  to  put  yet  another 
burden  to  the  DBMS  user.  In  addition,  there  will  always  be  old  programs  which 
will  not  have  such  specification  of  data  base  sets. 

Suppose  now  that  we  have  a  stream  of  transactions,  life  can  figure  out 
the  needs  of  each  one  as  a  set  of  segments.  The  needs  of  all  of  them  together  is 
obviously  the  aggregate  of  these  segments.  Such  a  union  can  easily  be  formed 
in  our  case.  In  this  way,  we  will  not  have  in  system  memory  a  copy  of  a  part  of 
the  data  base  for  each  transaction.  In  addition,  the  preprocessor  may  change 
the  order  of  transactions  and  group  similar  transactions  together  so  as  to  take 
advantage  of  the  residency  of  some  segments. 

There  are  some  DBMS  implementations  where  the  R,L,  and  /* s  are  not 
distinct  for  performance  reasons.  That  does  not  present  a  problem  providing  we 
know  exactly  where  they  are.  For  instance,  instead  of  having  a  separate  struc¬ 
ture  L  for  a  link  between  R  1  and  R2  we  can  implement  the  link  with  pointers 
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residing  in  R  1  and  R  2.  There  will  then  be  only  the  segments  for  R  1  and  R2  and 
we  will  not  need  another  segment  for  the  navigation.  The  same  reasoning  ap¬ 
plies  if  a  record  type  R  is  split  into  more  segments  for  performance  reasons.  We 
then  treat  these  segments  separately.  We  only  insist  that  the  mapping  from 
data  base  definition  to  data  base  implementation  be  reasonably  clean. 

6.  Concluding  Remarks 

We  have  discussed  the  problem  of  allocating  and  staging  data  in  a  large 
data  base.  Our  discussion  suggests  an  approach  in  which: 

1)  We  use  a  structured  virtual  space  to  encode  data  base  locality  in  the  paging 
scheme. 

2)  The  memory  manager  may  optionally  take  advantage  of  this  locality  to 
prepage. 

3)  We  use  an  LRU  replacement  policy  of  data  base  pages  in  processor  memory 
based  on  data  locality,  with  provisions  for  currency  indicators. 

4)  We  type  segments  according  to  the  data  base  definition  and  try  to  predict  and 
possibly  bring  into  the  system  memory  the  segments  that  a  transaction  may 
use. 

5)  We  use  an  LRU  policy  to  replace  segments  corresponding  to  programs,  system 
tables  and  indexes  in  system  memory  except  for  special  resident  segments. 

We  pose  far  more  questions  than  we  answer  in  this  paper.  Careful 
analysis  and  experimentation  may  prove  some  of  our  conjectures  wrong.  This  is 
unavoidable.  We  base  our  conclusions  on  arguments.  The  final  conclusions 
should  be  based  on  facts  and  numbers.  This  work  should  not  be  treated  as  a  pa¬ 
per  giving  final  answers  but  rather  a  paper  exploring  ideas.  We  hope  it  will 
stimulate  further  research  in  this  important  area. 
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Abstract 


This 

report 

investigates 

some  aspects 

of  the  effect 
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systems 
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Specificall y. 
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uman  factors 

aspects  of 

data  base  manage 

systems  is  considered.  The  focus  of  attention  is  the  inter 
between  the  user  and  the  DBMS.  The  evaluation  process  cons 
of  measuring  various  aspects  of  user  performance  via  h 
factors  experiments. 
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variables  that  affect  user  performance  within  the 
interface  are  identified  and  their  properties 
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discussed.  Next,  measures  for  determining  user  performance  are 
proposed  and  their  evaluation  discussed.  Two  user  performance 
experiments  that  compare  the  data  models  and  data  languages  of 
three  specific  DBMS's  the  EDBS  hierarchical,  DBTG-network ,  and 
relational  systems  -  are  discussed. 

In  the  first  experiment,  performance  is  measured  for 
different  types  of  users  in  combination  with  several  different 
applications  and  the  data  models  and  data  languages  of  the  three 
EEBS  DBMS's.  Two  user  types  and  three  different  applications  are 
used.  Subjects  are  required  to  code  and  implement  several  simple 
to  moderately  difficult  queries  for  each  application.  Coding 
accuracy,  coding  time,  and  debug  time  are  used  as  measures  of 
user  performance.  Some  significant  results  are  obtained  for  the 
coding  accuracy  measure  which  indicate  that  the  users  of  the  EDBS 
relational  system  produce  better  performance  levels  than  the 
users  of  the  other  two  systems.  Some  significant  results  for 
debug  time  are  also  obtained. 

In  the  second  experiment,  performance  is  measured  for 
different  types  of  users  in  combination  with  the  three  different 
EDBS  data  models  and  data  languages.  Two  user  types  are  used.  A 
single  application  is  used  for  which  subjects  are  required  to 
code  and  implement  several  simple  to  moderately  difficult 
queries.  Coding  accuracy,  query  comprehension,  and  query 
correction  are  used  as  measures  of  user  performance.  Significant 
results  are  obtained  for  the  coding  accuracy  measure  which 
indicate  better  performance  levels  for  the  users  of  the  EDBS 
relational  system. 
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The  errors  observed  in  the  two  experiments  are  analyzed  and 
their  source,  with  respect  to  the  data  models  and  data  languages, 
pinpointed.  Properties  of  the  data  models  and  data  languages 
that  adversely  affect  user  performance  are  discussed.  Changes 
are  proposed  to  eliminate  or  control  the  causes  of  the  errors 
observed.  Testable  performance  hypotheses  based  on  errors  that 
were  observed  are  formulated.  The  relevance  of  human  factors 
investigations  to  the  problems  of  DBMS  design  and  DBMS  selection 


is  discussed. 


Specification  and  Verification 

of 

Data  Base  Semantic  Integrity* 

by 

M.  L.  Brodie 


Department  of  Computer  Science** 
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Toronto,  Canada 


Abstract 

Semantic  integrity  is  fundamental  to  the  correct  application 
and  use  of  data  base  systems.  Two  conditions  necessary  for 
semantic  integrity  are:  the  logical  data  base  schema  must  be  a 
consistent  set  of  constraints  and  the  data  base  values  must 
satisfy  the  constraints  in  the  schema.  Unfortunately,  the 
techniques  currently  available  for  ensuring  semantic  integrity 
are  inadequate.  Constraint  specification  techniques  are 


*  This  report  is  available  as  CSF.G  Technical  Report  91  from  the 
Computer  Systems  Research  Group,  University  of  Toronto. 

**  Author’s  present  address:  Information  Systems  Management, 
University  of  Maryland,  College  Park,  Maryland  20742,  U.S.A. 
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incomplete  and  ad  hoc .  Due  to  the  absence  of  a  uniform  view  of 
constraints  and  semantic  integrity,  data  bases  are  difficult  to 
specify.  Few,  if  any,  techniques  exist  to  verify  that 
constraints  are  consistent  or  that  data  bases  satisfy  the 
appropriate  constraints.  Consequently,  the  application  of  data 
base  technology  is  severely  restricted. 

This  report  applies  techniques  from  programming  languages  and 
artificial  intelligence  to  the  specification  and  verification  of 
logical,  non-behavioural  properties  of  data  bases.  Abstract  data 
types,  specification  techniques,  abstraction,  and  data  modelling 
are  extended  for  the  particular  problems  of  data  bases,  notably, 
data  independence,  data  sharing,  problems  of  scale,  and  data 
semantics.  The  notion  of  a  data  type  algebra  is  developed  as  a 
basis  for  a  new  data  model.  The  data  model  and  a  corresponding 
specification  language  are  presented  and  axiomatized. 

A  major  difficulty  in  ensuring  the  semantic  integrity  of  a 
data  base  lies  in  ascertaining  whether  or  not  a  specification  is 
consistent.  The  axiomatically  defined  specification  language 
acts  as  a  uniform  basis  for  a  verification  technique  which 
extends  well  known  techniques  from  programming  languages. 

The  report  also  includes  discussions  on  both  the  semantics  of 
data  and  data  modelling.  A  data  base  design  methodology  based  on 
concepts  from  structured  programming  and  artificial  intelligence 
is  presented. 


Theory  of  Database  Mappings* 


by 

A.C.  Klug 
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University  of  Toronto 
Toronto,  Canada 


Abstract 

The  idea  of  a  mapping  is  important  within  data  base 
management;  most  state-of-the-art  database  management  facilities 
involve  mappings.  In  view  of  this  prominent  role  played  by 
mapping  concepts,  it  is  desirable  to  have  a  sound  theoretical 
foundation  from  which  to  base  further  research  in  advanced 
database  management  facilities.  This  report  contributes  to  the 
development  of  such  a  foundation. 

Mathematically  precise  definitions  of  structure  and  operation 
mappings  are  given.  In  this  formalism,  data  models  and  mappings 


*  This  report  is  available  as  CSRG  Technical  Report  98  from  the 
Computer  Systems  Research  Group,  University  of  Toronto. 
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resemble  abstract  machines  and  their  mappings.  Desirable 
structure  mappings  are  defined  to  be  the  ones  which  produce 
consistent  views  from  consistent  base  schemas.  Several  classes 
cf  desirable  operation  mappings  are  defined.  One  classification 
specifies  the  conditions  under  which  an  operation  is  correctly 
interpreted,  and  another  specifies  when  consistent  states  result 
from  the  operation  mapping.  We  also  compare  our  formalism  with 
two  others  from  the  literature,  and  we  compare  our  formalism  and 
the  predicate  calculus. 

Having  defined  the  desirable  properties  of  mappings,  we 
endeavor  to  give  necessary  and  sufficient  conditions  for  these 
properties  to  hold  within  a  relational  setting.  The  relational 
setting  contains  functional  dependencies,  equality  constraints 
and  relational  algebra.  Determining  desirable  structure  mappings 
is  shown  to  be  equivalent  to  generating  all  valid  constraints  on 
relational  algebra  expressions.  This  problem  is  undecidable  when 
set  difference  is  among  the  relational  algebra  operators.  By 
removing  this  operator,  we  have  a  decidable  problem,  and 
procedures  are  given  for  determining  the  ’'good*’  structure 
mappings. 

One  class  of  relational  operation  mappings  is  studied  in 
detail.  Recognizing  this  class  of  mappings  is  undecidable  in 
general,  but  a  restriction  analogous  to  the  one  above  produces  a 
decidable  problem,  and  necessary  and  sufficient  conditions  are 
given  for  this  restricted  problem. 
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Finally,  we  remark  on  how  the  conditions 
determining  the  various  types  of  mappings  within  a 
context  can  be  used  within  the  contexts  of  the  other 


given  for 
re lational 
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